Cost-Effectiveness of an Enhanced Whole-School Social Competency Intervention

Loading...
Thumbnail Image

Files

umi-umd-5810.pdf (334.2 KB)
No. of downloads: 1111

Publication or External Link

Date

2008-10-22

Citation

DRUM DOI

Abstract

Background

The cost and efficacy literature regarding elementary school-based preventative programs is limited, and many cost and efficacy studies suffer from research design and methodology problems.

Purpose

This study compares the marginal costs of a specific whole-school intervention to marginal costs of control school programs. It also assesses the cost-effectiveness of treatment and control interventions with respect to self-reported aggression, academic grades, and Maryland State Assessment (MSA) scores.

Setting

The study takes place in elementary schools in Anne Arundel County, a relatively diverse, suburban county in Maryland.

Subjects

Counselors (n=9) and principals (n=11) from 12 schools; the project manager, data clerk, and coordinator of guidance from the school system central office; and five research team members completed questionnaires about their time use. The study relies on efficacy data from another study.

Intervention

Implemented for three years, Second Step is a popular preventative, school-wide social competency program that aims to augment students' social skills and prevent problem behavior (Frey, Hirschstein, & Guzzo, 2000). Using manual-based lessons, classroom teachers in first through fifth grades were trained to deliver 30-minute lessons once a week to their classes in the areas of empathy, anger management, problem solving, and impulse control. Teachers also supplemented the formal lessons by reinforcing what had been taught at other times during the day (Frey et al., 2000). The intervention was enhanced by adding specific implementation standards and the use of periodic feedback about implementation to intervention managers and teachers.

Research Design and Methods

This study makes use of results from a large-scale randomized controlled trial that investigated the efficacy of Second Step. Researchers selected 12 elementary schools that had never implemented Second Step to participate, and these schools were matched based on their demographics and achievement history. Within each pair of schools, researchers randomly assigned one school to the treatment group and one to the control group. The methodology used for assessing costs is the ingredients approach (Levin & McEwan, 2001).

Data Collection and Analysis

Effect sizes for third and fourth graders for the third year are obtained from outcome evaluation reports. Third-year costs are added to retrospective training costs to estimate three-year costs. Data for costs of personnel time are collected in the form of time-use questionnaires, supplemental teacher questionnaires, and implementation logs. Accounting expenditures, rental agreements, and contacts with district personnel provide other cost data.

To reflect current and annual costs and to account for opportunity costs, costs are discounted (expressing future costs in terms of their present values) and amortized (distributing a cost across its lifetime). Finally, incremental cost-effectiveness ratios are calculated for some of the outcome measures examined. Sensitivity analyses are conducted to consider variability in cost and cost-effectiveness estimates.

Results

Results imply that the enhanced whole-school social competency intervention attained no positive effects in student self-reported aggression, academic grades and MSA scores, at a cost of only $69 less per student over a three-year implementation period.

Conclusions

The enhanced whole-school social competency intervention is no more cost-effective than the control programs: annual marginal student costs are only slightly less than those of the control program, and the program is not efficacious. Limitations include the small number of schools and personnel and correspondingly large standard errors for effect sizes, the use of self-report methods to estimate time, and dependence on unreliable accounting expenditure data from the school district. Undependability of cost data may result from both measurement error and bias. In addition, since this particular Second Step program was implemented in only one school district, its generalizability to other school districts or variations of program implementation is unexplored.

Despite these limitations, the study provides a range of credible values for cost-effectiveness for the program. It may provide insight to the scientific community about the costs involved in operating an enhanced whole-school intervention to share with school administrators and educators in their considerations of elementary school-based preventative interventions.

Notes

Rights