Developing and validating a measure of epistemic competence beliefs to examine undergraduate students’ critical-analytic thinking in a multiple source use task

Abstract

Background: This dissertation aimed to develop and validate a novel Epistemic Competence Beliefs Measure (ECBM) to capture students’ ability to identify and utilize relevant sources for complex issues. Epistemic competence was hypothesized to be critical in performing multiple source use (MSU) tasks, particularly in predicting critical-analytic thinking in argumentative essays. The study was conducted in two phases, focusing on the ECBM’s development, its content validity, and predictive validity in an MSU context.

Methods: Phase 1 involved creating the ECBM’s based on epistemic beliefs and cognition theories, presenting students with controversial scenarios. Content validity was assessed by an international panel of experts. Phase 2 implemented the ECBM in a university course, collecting data through argumentative claim selection forms, search logs, notes, essays, and a retroactive behaviors questionnaire. Data were analyzed using content analysis, cluster analysis, ANOVA, multiple linear regression, and regression trees to determine the ECBM’s predictive validity.

Findings: The study revealed significant variability in students’ epistemic competence as measured on the ECBM, though no direct predictive relationship to enacted epistemic competence and exhibited critical-analytic thinking was established. Students’ critical-analytic thinking varied significantly, influenced by their GPA and TORR scores. Notably, students with higher relational reasoning abilities exhibited superior critical-analytic thinking in their essays, supporting the theorized link between these constructs.

Implications: For future research, the ECBM can be refined and more closely integrated into the MSU project by aligning its completion with the task’s introduction. This integration may enhance students’ epistemic agency and awareness. Furthermore, diversifying study populations across different sociocultural contexts and employing Bayesian and mixed-methods analyses can provide deeper insights into epistemic competence and critical-analytic thinking. Practical implications suggest procedural adjustments to better align the MSU project with theoretical frameworks, potentially improving instructional practices.

Conclusions: The novel Epistemic Competence Beliefs Measure is a meaningful contribution to the literature on epistemic beliefs as it unearthed theoretically and practically meaningful profiles of undergraduates’ appraisal of the characteristics of complex, controversial issues. The data-analytic focus on students’ variability rather than only consistency in characterizations of the scenarios highlighted the value of examining epistemic beliefs in a more situated, contextualized manner. This resulting findings of varying beliefs dispute the lingering assumption that epistemic beliefs are stable across contexts. While the assessment of the ECBM’s predictive validity identified no significant relations, the findings underscore the importance of relational reasoning to critical-analytic thinking. Future research should focus on refining the ECBM, exploring its applicability in diverse contexts, and employing comprehensive analytical methods to further elucidate these constructs.