College of Education
Permanent URI for this communityhttp://hdl.handle.net/1903/1647
The collections in this community comprise faculty research works, as well as graduate theses and dissertations..
Browse
20 results
Search Results
Item UNDERSTANDING POLICY ENACTMENT IN DIVERSE INSTITUTIONAL CONTEXTS: AN EXPLORATION OF THE IMPLEMENTATION OF THE SINGLE NATIONAL CURRICULUM POLICY IN PAKISTAN(2023) Ali, Fatimah; Klees, Steven J; Education Policy, and Leadership; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Education reform is a contested and complicated terrain where multiple stakeholders with competing visions seek to influence the direction of reform in alignment with their own interests. As a result of these competing visions, education policy is not value free or neutral and favors those who hold the political and economic power to maintain the inertia of the status quo. Despite this unequal balance, education reform remains a dialectic process which is affected by several factors including contexts, resources, accountability processes and the agency of those tasked with policy implementation. Since policy is rarely implemented without some refraction from its original formulation, it is important to examine the implementation process to understand its complexities that can aid in better policymaking by decisionmakers. This multi-case qualitative study examines the implementation of the Single National Curriculum (SNC) Policy introduced during the 2021-2022 academic year in Punjab, Pakistan in primary schools (grade 1-5). Using an institutional lens, this study investigates how public and private schools interpreted the Single National Curriculum Policy and implemented it in classrooms. To highlight the complexity of policy implementation and provide a variegated and nuanced view of policy implementation within different institutional and social contexts, the study was conducted in a public school, a low-cost private school, and a high-cost private school in the cities of Lahore and Kamalia. In total, 22 teachers and administrators across the three schools were interviewed for this research. This study found that the alignment of belief systems, the institutional arrangements of schools and perceived costs of deviation all influenced how closely the SNC curriculum was implemented in each school. Schools interpreted the policy in light of mediating factors which included institutional factors such as the learning environment, capacity and agency of the teachers and administrators as well as their hierarchies and support systems. Adherence to policy was also dependent on the perceived risk of being inspected by government officials and monitoring frameworks established to assess policy implementation. Lastly, in all three schools, administrators and teachers adhered to a spectrum of beliefs that were in alignment with or opposed to the SNC policy. These governed how closely the SNC was implemented in classrooms and or if corrective pressure was applied to maintain their desired student outcomes. As a result of this variation, gaps in student learning remained intact as the new policy did not address the roots of quality differences across schools. However, the government was able to successfully coerce all schools into incorporating more religious instruction and nationalistic citizenship education leading to more tight coupling between the schools on this front.Item Characterizing the Adventitious Model Error as a Random Effect in Item-Response-Theory Models(2023) Xu, Shuangshuang; Liu, Yang; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)When drawing conclusions from statistical inferences, researchers are usually concerned about two types of errors: sampling error and model error. The sampling error is caused by the discrepancy between the observed sample and the population from which the sample is drawn from (i.e., operational population). The model error refers to the discrepancy between the fitted model and the data-generating mechanism. Most item response theory (IRT) models assume that models are correctly specified in the population of interest; as a result, only sampling errors are characterized, not model errors. The model error can be treated either as fixed or random. The proposed framework in this study treats the model error as a random effect (i.e., an adventitious error) and provides an alternative explanation for the model errors in IRT models that originate from unknown sources. A random, ideally small amount of discrepancy between the operational population and the fitted model is characterized using a Dirichlet-Multinomial framework. A concentration/dispersion parameter is used in the Dirichlet-Multinomial framework to measure the amount of adventitious error between the operational population probability and the fitted model. In general, the study aims to: 1) build a Dirichlet-Multinomial framework for IRT models, 2) establish asymptotic results for estimating model parameters when the operational population probability is assumed known or unknown, 3) conduct numerical studies to investigate parameter recovery and the relationship between the concentration/dispersion parameter in the proposed framework and the Root Mean Square Error of Approximation (RMSEA), 4) correct bias in parameter estimates of the Dirichlet-Multinomial framework using asymptotic approximation methods, and 5) quantify the amount of model error in the framework and decide whether the model should be retained or rejected.Item THE EFFECT OF DIFFERENT RELATIVE LOGISTIC REGRESSION GENERATED PROPENSITY SCORE DISTRIBUTIONS ON THE PERFORMANCE OF PROPENSITY SCORE METHODS(2020) An, Ji; Stapleton, Laura M; Human Development; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Much education research involves evaluating the causal effects of interventions. The propensity score (PS) method, which is often used to account for selection bias, has become a popular approach to facilitating causal inference in quasi-experimental designs. Because the success of the application of PS conditioning methods is dependent on the estimated propensity scores, the relative PS distribution between the treated and control groups could be an important yet not well-known factor. The primary goal of this dissertation was to explore, via a simulation study, the relations between the relative PS distributions and the performance of selected PS matching methods. The results indicated that PS weighting (without trimming) tends to be robust to a variety of data conditions and produces more accurate and trustworthy TE and SE estimates. The performance of the methods and conclusions were then illustrated through an empirical data analysis using data selected from the Early Childhood Longitudinal Study Kindergarten Class of 2010-11 study, assessing the effect of having home computers on first grade students’ math achievement.Item The Performance of Balance Diagnostics for Propensity-Score Matched Samples in Multilevel Settings(2019) Burnett, Alyson; Stapleton, Laura M; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The purpose of the study was to assess and demonstrate the use of covariate balance diagnostics for samples matched with propensity scores in multilevel settings. A Monte Carlo simulation was conducted that assessed the ability of different balance measures to identify the correctly specified propensity score model and predict bias in treatment effect estimates. The balance diagnostics included absolute standardized bias (ASB) and variance ratios calculated across the pooled sample (pooled balance measures) as well as the same balance measures calculated separately for each cluster and then summarized across the sample (within-cluster balance measures). The results indicated that overall across conditions, the pooled ASB was most effective for predicting treatment effect bias but the within-cluster ASB (summarized as a median across clusters) was most effective for identifying the correctly specified model. However, many of the within-cluster balance measures were not feasible with small cluster sizes. Empirical illustrations from two distinct datasets demonstrated the different approaches to modeling, matching, and assessing balance in a multilevel setting depending on the cluster size. The dissertation concludes with a discussion of limitations, implications, and topics for further research.Item TEACHERS’ PERCEPTIONS OF EDTPA ON THEIR PRACTICE(2018) Seelke, John Louis; Brantlinger, Andrew; Education Policy, and Leadership; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)For centuries, education policymakers have sought to identify the most effective way to assess a potential teacher’s readiness to enter the classroom. These assessments evolved from multiple choice examinations to performance-based assessments focused on teacher actions. The latest iteration of these performance-baed assessments is edTPA. edTPA’s structure mirrors that of the assessment for National Board Certification (NBC) designed for veteran teachers. The NBC assessment has shown to be educative for teachers who complete it, leading to positive changes in their post-assessment practice (Athanases, 1994; Hattie & Clinton, 2010; Sato, Darling-Hammond and Wei, 2008; Steeley, 2003). This study examines whether edTPA has similar educative impacts on early career teachers. Since edTPA is relatively new, little research has been completed on its impact on teacher practice. Most of the current literature on edTPA focuses on its implementation or on pre-service candidate perceptions of completing the assessment. This interview study also examines candidate perceptions but focuses on whether they felt completing edTPA was educative and impacted their current practice. This study includes twenty teachers who participated in two hour-long interviews given roughly six months apart. All of the participants were recent secondary mathematics education graduates from one university. This study is among the first studies of edTPA to include teachers who both completed edTPA and have been teaching for at least two years. Another unique strength of this study is that, prior to the second interview, candidates reviewed their actual edTPA portfolio to help recall components of the assessment and to potentially make clearer connections between edTPA and their current practices. The study results support the notion that edTPA can be educative and influence a teacher’s current practices around planning, instruction, and assessment. The level of influence that completing edTPA has on a teacher’s practices may be impacted by school or district policies that either hinder or support high-scoring edTPA practices. The results demonstrate how edTPA can not only be seen as a summative tool at the end of pre-service teaching, but also a formative tool that impacts the teaching practices of early career teachers.Item An Analysis of a Project Management Oversight Committee as a School Improvement Model in One School District(2018) Alston, Anthony Reginal; Imig, David G; Education Policy, and Leadership; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)There are persistent and significant gaps in performance between various racial and ethnic sub-groups, particularly between African American and White groups of students, on both state assessments and discipline measures (Rothstein, 2014). Effectively closing the gap requires improving the performance of all students while accelerating the achievement of low-performing student groups in order to catch up to their higher-performing peers. Researchers have found that a variety of school, community, economic and familial factors correlate with low student performance and the achievement gap, but views are mixed on how to improve performance for all and narrow the gap (Alliance for Excellence Education, 1999; Cancian & Danziger, 2009; Jacob & Ludwig, 2009; Janet, 1999; Mitra, 2011; Organization for Economic Co-Operation (OECD), 2012). Over the past decade, federal, state and local policies have made the improvement of low-performing schools and the closing of the achievement gap a top priority. This descriptive quantitative study focused on the efforts of one Mid-Atlantic system to address low performance among certain groups of students using a Project Management Oversight Committee (PMOC) model. The model utilizes data and a heightened level of accountability to yield quick, yet sustainable growth and progress. For this study seven schools (three high schools, two middle schools and two elementary schools) were identified as a result of persistent participation in the process. The academic and discipline data were reviewed to determine trends in performance. In addition to the achievement and demographic data, a satisfaction survey was reviewed to understand the impact of the PMOC process in improving low student performance. Analysis of the data did not find any relational impact upon achievement data. Although the PMOC process did not have the promised impact, it does not deminsh the possibilty that there were gains that would not have occured if the process had not been used. Based on these findings,it is recommended that the system consider allocating funding for further studies to examine the impact of the PMOC process.Item TEACHERS’ PERCEPTIONS OF HOW THE USE OF PEER EVALUATION COULD IMPROVE THEIR TEACHING PRACTICE(2017) Farrell, Dale Patrick; Richardson, Patricia M; McLaughlin, Margaret J; Education Policy, and Leadership; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)As the method, instrument, and reliability of teacher and principal evaluations has come under extreme scrutiny since the Education Reform Act of 2010, school systems across the nation have examined and refined the evaluation process for teachers and principals. Studies have shown the integration of peer evaluation as a model of teacher evaluations can have a positive impact on the teachers and their performance that participate in peer evaluation as well as the potential for an increase in the academic achievement of students. The purpose of this study was to investigate teachers’ overall perceptions of peer evaluation and of how the use of peer evaluation could improve their teaching practice. The goals of this study were to determine the level of sharing of instructional practices among their peers currently, and whether or not they perceived an increase in the amount of time spent sharing of best practices would occur as a result of incorporating peer evaluation into the current model of a formative evaluation. Further, teachers were asked to identify potential benefits as well as any potential challenges they see as a result of implementing peer evaluations. This study used an online survey to gather data from participants. This study was conducted in a suburban school district in Maryland. A total of 34 teachers participated in the survey questionnaire. One elementary, one middle, and one high school were selected to participate in the survey. Data were collected through an online survey conducted in September 2017. This study provides some evidence that teachers do welcome the possible integration of peer evaluation and perceive that peer evaluation may result in an increase in the amount of time spent sharing instructional strategies among other teachers. The findings also reported potential benefits such as increased sharing of best practices of instructional strategies, more timely and relevant feedback, and reduced feelings of isolation could potentially occur as a result of incorporating peer evaluation in the current model of formative evaluations. This study further identified potential challenges such as lack of time to complete peer evaluations, personal bias, and a perceived feeling of needing to add yet another task to the busy professional day of a teacher.Item TEACHER PERCEPTIONS OF TEACHER EVALUATION USING THE TEACHER PERFORMANCE ASSESSMENT SYSTEM AND FACTORS THAT CONTRIBUTE TO TEACHER QUALITY, PROFESSIONAL GROWTH, AND INSTRUCTIONAL IMPROVEMENT OVER TIME(2017) Jaffurs, Alex; Richardson, Patricia M; Education Policy, and Leadership; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The primary purpose of teacher evaluation is to improve teaching practice, which results in increased student achievement. In practice, however, evaluation systems have been generally used as sorting mechanisms for identifying the lowest performing teachers for selective termination. The school system in this study, like others, aspires to have all of its teachers consistently performing at a highly effective level. The problem of practice faced by the school system is the inability of a large number of teachers rated “effective” to summarily improve their practice over time and move to the “highly effective” rating. In essence, how does a teacher evaluation metric maximize the chances that those who remain in the profession become accomplished practitioners? This research triangulates teacher evaluation, self-reflection and their roles in improving teacher quality. The prevailing thought is that teachers who willingly engage in more formalized self-reflection and self-assessment yield higher degrees of teacher effectiveness as measured on a local teacher evaluation. The central focus of this study will investigate tenured teachers’ perceptions of the effect of their teacher evaluation tool on teacher quality and other factors that contribute to a teacher’s improvement of instructional performance over time. The researcher would also like to investigate the extent to which teacher cohorts – differentiated by demographic data - engage in formalized practices of self-reflection about their own teaching practice. Lastly, the researcher would like to determine whether or not tenured teachers who are evaluated with the local teacher evaluation tool actually improve their teacher effectiveness over time. This study was conducted in a public, K-12 school system with 1420 teachers employed - 39 of which are National Board Certified. This schools system is located in a rural/suburban school system and has utilized its current teacher evaluation system since 2000. The findings of this study indicated that the majority of teachers – disaggregated by demographic teacher cohort - viewed their local teacher evaluation system somewhere along the continuum of neutral to satisfactory as a tool for building a teacher’s effectiveness over time. The overwhelming majority of teachers embraced the post-conference as the most impactful part of the entire evaluation process in building teacher quality; the least impactful was the pre-conference. Additionally, teacher respondents – agnostic of demographic – opined that while the local teacher evaluation system was perceived to be a both quality control and a compliance factor for teachers, less than half of all respondents believe that the system, assists teachers formatively as a tool for professional development. Per the respondents, it should be noted that the teacher evaluation system elicited the strongest reactions – both positive and negative - in teachers having experienced more than 20 formal observations. The research also conveyed that most teachers reported that there was much more embedded self-reflection in the evaluation system than hypothesized; most prominently, teachers cited that audio-taping, reviewing student performance data, completing a self-reflective checklist, and engaging in unstructured self-reflection were a few of the assorted self-reflective activities were facilitated by the evaluation system. Moreover, the data clearly demonstrated that all teachers engage in high degrees of reflection regardless of demographic cohort and a majority of teachers claim to already know how to “self-reflect.” In other words, the highest self-reported degree of reflection were those teachers already rated as “highly effective” in the local evaluation system. A prevalent trend in the data was that degrees of self-reflection matter and build more pronounced levels of teacher effectiveness over time. In essence, the fact that teachers participate in reflection does not seem to impact teacher quality; rather, the degree and amount to which one reflects is actually what matters in building instructional capacity in teachers. Other noticeable trends in the data were as follows: more years of teaching experience was inversely related to the degree to which a teacher self-reflects; over 30% of teachers with more than 20 years of experience reported that that they do no self-reflect at all; the non-NBCT teacher cohort out reflects the NBCT cohort; NBCT teachers had the highest average evaluation rating out of every teacher cohort; and, teaching experience seems to mute any lack of reflection in a teacher’s evaluation rating; The other noticeable trend was that more formal observations for teachers did not translate into higher evaluation ratings over time. Overall, the two most impactful professional development activities cited by teachers were the following: participation in professional learning communities and peer coaching and mentoring, respectively.Item Measuring Transactive Memory Systems Using Network Analysis(2017) King, Kylie Goodell; Stapleton, Laura M.; Sweet, Tracy; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Transactive memory systems (TMSs) describe the structures and processes that teams use to share information, work together, and accomplish shared goals. First introduced over three decades ago, TMSs have been measured in a variety of ways. This dissertation proposes the use of network analysis in measuring TMS. This is accomplished by describing the creation and administration of a TMS network instrument, evaluating the relation of the proposed network measures and performance, and considering the validity of the proposed network measures. Although the proposed network measures do not appear to be valid in their current form, this study provides motivation for future exploration of using instrumental networks as measures of TMS.Item An Investigation of Middle Grade Mathematics and Language Arts Teachers’ Knowledge and Beliefs of Student Learning Objectives(2017) Miller, Zipporah Miller; McLaughlin, Margaret; Norris, John; Education Policy, and Leadership; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Teacher evaluations in most school districts, until recently, have served as an exercise in compliance rather than a means to identify the strengths and weaknesses of each teacher in order to provide appropriate support to improve professional practice and increase student learning. Current federal legislation has stimulated districts to rethink their evaluation systems. In Saint Lawrence School District (SLSD) we discovered a misalignment between middle grade student performance in mathematics and reading and teacher ratings in those schools. Although over 50% of the students failed to meet standards in mathematics and reading, the majority of teachers were rated as highly effective. The SLSD recently adopted a new evaluation system that includes multiple measures to indicate teacher strengths and weaknesses and identify the support needed. Student Learning Objectives (SLOs) are a major component of the new system, added to measure the individual teacher’s contribution to student learning. This study investigated middle grade mathematics and language arts teachers’ knowledge and understanding of the intent and processes associated with the implementation of the Student Learning Objectives; the extent to which teachers report that student data from the SLO process have changed their instructional practices; and teacher perceptions regarding the value of SLOs in improving student learning and their own professional growth. Of the 315 middle school teachers invited to participate in the study, 232 teachers from the 19 comprehensive middle schools in SLSD submitted an anonymous electronic survey. The study incorporated exploratory design and used descriptive statistics, graphic presentations, and significant tests to examine patterns, behaviors, and viewpoints of those teachers. The results indicate that participants understand the intent and process of SLOs, but perceive no noticeable changes in teacher practice as a result of SLO implementation. Although most teachers agree that professional development needs should be based on the needs of students, over 50% disagree with the SLO process as a means to improve their teaching. Teachers with fewer years at their current school reported they had a greater belief in the use of student data to improve instruction than their counterparts with over four years at their school.