The movement to use empirically supported treatments has increased the need for researchers and supervisors to evaluate therapists’ adherence to and the quality with which they implement those interventions. Few empirically supported approaches exist for providing these types of evaluations. This is also true for motivational interviewing, an empirically supported intervention important in the addictions field. This study describes the development and psychometric evaluation of the Motivational Interviewing Supervision and Training Scale (MISTS), a measure intended for use in training and supervising therapists implementing motivational interviewing. Satisfactory interrater reliability was found (generalizability coefficient p2 = .79), and evidence was found supporting the convergent and discriminant validity of the MISTS. Recommendations for refinement of the measure and future research are discussed