Examining evidence of reliability and validity of mental health indicators on a revised national survey measuring college student health

Abstract

The American College Health Association-National College Health Assessment (ACHA-NCHA), which began surveying college students in 2001, is currently the only large scale survey available for colleges and universities to measure a variety of health constructs among their student populations. Beginning in December 2005, the survey underwent an extensive revisions process in an effort to improve its measurement quality and to better capture the current health status of college students. Revisions were guided by changing student health priorities, feedback from respondents, literature focused on characteristics of reliable and valid survey questions, and the Model of Survey Response. As mental health concerns continue to rise on campuses today, this content area of the ACHA-NCHA was dramatically expanded, as it now includes constructs such as diagnosis and treatment with multiple mental health conditions, difficult life experiences, stress, and help-seeking. This dissertation, which is a secondary data analysis of data collected from the original and revised ACHA-NCHA surveys during an experimental field pre-test of the modified survey, documents the survey revisions process, provides results from more than 40 mental health indicators by various demographic characteristics, and establishes the reliability and validity of the mental health indicators. The data analyzed in this study were collected from students from 7 U.S. college and universities who were randomized to complete either the original (final N = 6,216) or the revised (final N = 6,110) online ACHA-NCHA from February through May 2007. It was hypothesized that (1) changes to survey indicators designed to measure comparable constructs would result in significant differences in student response patterns across versions of the ACHA-NCHA; (2) mental health indicators on the revised survey would demonstrate evidence of internal consistency reliability, construct-related validity, and criterion-related validity; and (3) modified mental health indicators on the revised ACHA-NCHA would demonstrate greater evidence of reliability and validity than comparable indicators on the original ACHA-NCHA. Findings from this study at least partially support all hypotheses, and the revised ACHA-NCHA demonstrates preliminary evidence that is it a psychometrically sound survey tool to measure college student mental health constructs.