Examining the Multidimensional Nature of Engagement: The Development of the Multidimensional Engagement Rubric (MER)

Abstract

The current literature on the construct of engagement lacks clarity in how it defined, the dimensions associated with engagement, and the way it has been measured. Establishing agreement on how engagement is conceptualized and operationalized will inform the development of instruments that measure this construct accurately. Engagement is an important topic of study given its relationship to students’ reading achievement. Understanding the engagement needs of growing populations of students in K-12 schools, such as English Learners (ELs), is vital to their academic success. Therefore, the purpose of this dissertation is to clarify the conceptual and operational problems with the construct of engagement, assess the content validity of a newly developed engagement instrument through experts’ judgements, and test the reliability and criterion-related validity of this instrument with a small sample of ELs.

Chapter 2 is a research synthesis that examined the discrepancies surrounding the construct of engagement using instruments that measure engagement for upper elementary and middle school students. Two empirical studies comprise Chapters 3 and 4 of this dissertation. Study 1 assessed the content validity of the items included in the Multidimensional Engagement Rubric (MER) through expert opinion. Study 2 explored the reliability and validity of this instrument when used to measure the engagement of 6th grade English Learners (ELs) participating in literacy instruction.

The results from these three papers have uncovered several findings. First, the instruments used to measure student engagement have been constructed using a confounded body of literature which may, in turn, have led to the development of measures that may not have assessed this construct with precision. Next, the MER was developed after synthesizing the engagement literature and eliciting feedback from experts on engagement and motivation, which revealed a need for revising indicators included in the MER. Evidence of high reliability was revealed through weighted Kappa analysis, while criterion-related validity of the MER, when correlated with time on task scores, revealed moderate positive correlations. Finally, correlations between engagement and participants’ ages revealed mostly weak relations despite the small sample size.