In today's data driven world, the development of a statistically literate society is critical. As a result, many students are enrolling in university level introductory statistics courses and educators are promoting the development of strong understandings of the material taught in those courses. Statistical hypothesis testing, a powerful method of inferential statistics widely used in research, is taught in introductory courses. Though algorithmic in nature, statistical hypothesis testing is based on statistical theory. It is important that introductory students develop connected understandings of the algorithm, the concepts and logic that support it, and its uses.

This study explored the degree to which undergraduate, introductory statistics students develop desired understandings of the overall "big picture" of statistical hypothesis testing.  In order to investigate student understanding a mixed methods approach was employed--both large scale quantitative and small scale qualitative data were collected.  In the quantitative phase, a framework for assessing understanding of the conceptual and logical foundations of statistical hypothesis testing and its uses was created, a multiple-choice instrument with items representative of the framework was constructed, and data on student performance on this instrument were collected. Scores from a course exam that assessed student ability to use the algorithm to solve traditional statistical hypothesis testing problems were collected and compared with those from the multiple-choice instrument.  In the qualitative phase, in order to gain more insight into student thinking, follow-up interviews were conducted with students who represent a range of performance patterns on the two quantitative assessments.

The data collected in this study indicated that introductory statistics students do not develop strong, connected understandings of the "big picture" of statistical hypothesis testing.  Though they are able to perform the procedures, students do not have strong understandings of the concepts, logic, and uses of the method.  A weak correlation between scores on the quantitative assessments indicated that procedural knowledge is not a predictor of overall understanding of statistical hypothesis testing.  Analysis of quantitative and qualitative data indicated that students do not understand the role of indirect reasoning and inference in implementing and interpreting the results of a statistical hypothesis test.