Developing characterizations of problem-solving processes, strategies, and challenges from process and product data in digitally delivered interactive assessments: case study.

Simple item page
dc.contributor.advisorHarring, Jeffreyen_US
dc.contributor.authorCaliço, Tiago Alexandreen_US
dc.contributor.departmentHuman Developmenten_US
dc.contributor.publisherDigital Repository at the University of Marylanden_US
dc.contributor.publisherUniversity of Maryland (College Park, Md.)en_US
dc.date.accessioned2019-09-25T05:32:17Z
dc.date.available2019-09-25T05:32:17Z
dc.date.issued2019en_US
dc.description.abstractGames and simulation-based assessments (GSBAs) are the focus of increased interest in educational assessment given their ability to operationalize assessment tasks that mimic real world scenarios. Combined with the capacity to unobstrusively collect data on task-solving behavior, sometimes referred to as \emph{process} or \emph{event} data, GSBAs have the potential to expand the scope and nature of inferences about students' skills, knowledge, and abilities. A case study and a simulation study explored the viability of using concepts and analytical tools from the field of Business Process Mining (BPM) to facilitate the generation of evidence identification rules from behavioral, event-based data generated in the context of a GSBA. The case study demonstrate the utility of a process guided by the principles of Evidence-Centered Design (ECD) in order to define and refine Student, Task and Evidence Models. The BPM conceptual and analytical tools allowed to economically investigate the feasibility of using aspects of task-solving behavior, such as differences in targeted event sequences, as evidentiary sources. Bayesian Networks were then use to aggregate traditional score data with behavioral data in order to predict student membership to latent classes. Given the novel nature of the analytical method used to identify evidence rules, known as the Fuzzy Miner, a simulation study investigated the impact of sample size, expert classification of a training sample, behavioral variability, and modeling parameters in the ability of the method to identify differences in process structure across groups. The simulation results show that the method's robustness to several sources of noise, suggesting its utility as an exploratory tool to be integrated with expert judgment when generating evidence identification rules.en_US
dc.identifierhttps://doi.org/10.13016/xyvq-vsbh
dc.identifier.urihttp://hdl.handle.net/1903/24902
dc.language.isoenen_US
dc.subject.pqcontrolledEducational tests & measurementsen_US
dc.subject.pquncontrolledBayes Netsen_US
dc.subject.pquncontrolledBusiness Process Modelingen_US
dc.subject.pquncontrolledFuzzy Mineren_US
dc.subject.pquncontrolledGame and simulation based assessmenten_US
dc.subject.pquncontrolledprocess dataen_US
dc.titleDeveloping characterizations of problem-solving processes, strategies, and challenges from process and product data in digitally delivered interactive assessments: case study.en_US
dc.typeDissertationen_US

Files

Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
CalixE7o_umd_0117E_20176.pdf
Size:
3.25 MB
Format:
Adobe Portable Document Format
Download
(RESTRICTED ACCESS)

Collections

UMD Theses and Dissertations
Human Development & Quantitative Methodology Theses and Dissertations