Teachers affect student achievement. Measuring what makes teachers "effective" is a challenge without a clear definition of the construct or constructs involved. Self-reports cost little and allow for data collection from large samples, but the reliability and validity of self-report measures for studying teacher effectiveness have not been adequately examined. This study explored the utility of a self-report measure of instruction (Instructional Practices Scale). Hierarchical linear modeling was used to examine the effects of the scale on students' reading and math standardized test scores and report card grades. Although the scale showed small to moderate relationships with teacher characteristics, results suggested little predictive validity and little discriminant validity. Further, the effects of teacher-reported instruction on achievement were not dependent on students' entering level of achievement. When measuring loosely defined constructs such as "effective instruction," the cost of using a self-report measure may outweigh the benefits.