Instrument Development For Continuing Medical Education Evaluation

Abstract

The purpose of this study was to develop a valid, reliable and adaptable CME evaluation instrument to facilitate the future CME evaluation effort as well as contribute to the literature of CME evaluation studies. A generic instrument template was first developed addressing variables in the second evaluation level based on the TPB, i.e. attitude, behavioral belief, subjective norm, perceived behavioral control and behavioral intention. The instrument was then adapted to a CME-related conference, Preoperative Therapy in Invasive Breast Cancer: Reviewing the State of the Science and Exploring New Research Directions. Data were collected at the conference. A total of 134 physicians returned their questionnaires. Principle axis factoring with oblique rotation was used to examine the underlying structure of the data and reduced the items in the instrument to six subscales: positive beliefs, negative beliefs, subjective norms, perceived behavioral control and behavioral intention. Factor loadings supported the existence of six valid scales. The consistency between the a priori subscales and the factors emerged served as evidence for content validity of the instrument. Overall, all the subscales had sufficient reliability (alpha>= 0.70) for early stage instrument development showing the unidimensionality of the subscales. Scale modifications based on item analyses were conducted. The problematic items were eliminated, and the analyses were rerun. A 22-item instrument and a revised generic instrument template were finally developed. This study determined the adaptability of the theory based instrument template to the NCI CME conference and the feasibility of developing a content specific, valid and reliable CME evaluation instrument from the template assessing the changes in the concepts listed in the second evaluation level. The established and validated instrument could further be used to evaluate the effectiveness of other CME activities having the template adapted to different clinical domains addressed by each individual CME activity.