Coding Counsellor Behaviour in Motivational Interviewing Sessions: Inter‐Rater Reliability for the Swedish Motivational Interviewing Treatment Integrity Code (MITI)

The aim of this study was to evaluate inter‐rater reliability when using the Swedish version of the Motivational Interviewing Treatment Code (MITI) as an adjunct to MI training, clinical practice and research. Coders were trained to use the MITI for scoring taped sessions. The 4‐month basic training had a duration of 39 hours. Following training, 60 audio‐taped live interviews were randomly assigned for MITI coding. Mean intra‐class correlation (ICC) coefficients were calculated for 7 coders across all pairs of coders. Cronbach's alpha was calculated to estimate the covariance between each pair across their common interviews. Six months later, a second inter‐rater reliability test was performed, when 5 coders coded the same 15 randomly selected tapes. At the second reliability testing the mean ICC was 0.81 and the mean Cronbach's alpha was 0.96. However, the ICC varied for different sub‐variables of the MITI, ranging from 0.42 empathy to 0.79 for number of Closed questions. In conclusion, MITI shows promising potential to be a reliable tool to confirm and enhance MI training as well as practice in clinical settings and in evaluating MI integrity in clinical MI research. However, coder assessment of empathy and MI‐spirit, “global” variables, requires further refinement.     

Development and Initial Validation of the Assessment of Motivational Interviewing Groups – Observer Scales (AMIGOS)

This article documents the development and initial validation of a global observer instrument, the Assessment of Motivational Interviewing Groups – Observer Scales (AMIGOS). The AMIGOS includes three scales composed of 18 items documenting group processes, client-centered style, and motivational interviewing (MI) change focus. Three experienced coders independently rated 18 sessions of various group approaches. The AMIGOS showed high inter-rater intraclass correlation coefficient (ICC) and Cronbach’s alpha, strong convergent and discriminant validity with the MI Treatment Integrity scales (MITI), the Therapist Empathy Scale (TES), and the Group Climate Questionnaire (GCQ). The AMIGOS shows promise as a reliable and valid measure of MI group leadership and group processes, and could be useful measuring other group therapies as well.     

A Test of the Validity of the Motivational Interviewing Treatment Integrity Code

To evaluate the Swedish version of the Motivational Interviewing Treatment Code (MITI), MITI coding was applied to tape‐recorded counseling sessions. Construct validity was assessed using factor analysis on 120 MITI‐coded sessions. Discriminant validity was assessed by comparing MITI coding of motivational interviewing (MI) sessions with information‐ and advice‐giving sessions as well as by comparing MI‐trained practitioners with untrained practitioners. A principal‐axis factoring analysis yielded some evidence for MITI construct validity. MITI differentiated between practitioners with different levels of MI training as well as between MI practitioners and advice‐giving counselors, thus supporting discriminant validity. MITI may be used as a training tool together with supervision to confirm and enhance MI practice in clinical settings. MITI can also serve as a tool for evaluating MI integrity in clinical research.     

A test of the validity of the motivational interviewing treatment integrity code

To evaluate the Swedish version of the Motivational Interviewing Treatment Code (MITI), MITI coding was applied to tape-recorded counseling sessions. Construct validity was assessed using factor analysis on 120 MITI-coded sessions. Discriminant validity was assessed by comparing MITI coding of motivational interviewing (MI) sessions with information- and advice-giving sessions as well as by comparing MI-trained practitioners with untrained practitioners. A principal-axis factoring analysis yielded some evidence for MITI construct validity. MITI differentiated between practitioners with different levels of MI training as well as between MI practitioners and advice-giving counselors, thus supporting discriminant validity. MITI may be used as a training tool together with supervision to confirm and enhance MI practice in clinical settings. MITI can also serve as a tool for evaluating MI integrity in clinical research.     

Reliability and Validity of a Treatment Fidelity Assessment for Motivational Interviewing Targeting Sexual Risk Behaviors in People Living with HIV/AIDS

This study psychometrically evaluates the Motivational Interviewing Treatment Integrity Code (MITI) to assess fidelity to motivational interviewing to reduce sexual risk behaviors in people living with HIV/AIDS. 74 sessions from a pilot randomized controlled trial of motivational interviewing to reduce sexual risk behaviors in people living with HIV were coded with the MITI. Participants reported sexual behavior at baseline, 3-month, and 6-months. Regarding reliability, excellent inter-rater reliability was achieved for measures of behavior frequency across the 12 sessions coded by both coders; global scales demonstrated poor intraclass correlations, but adequate percent agreement. Regarding validity, principle components analyses indicated that a two-factor model accounted for an adequate amount of variance in the data. These factors were associated with decreases in sexual risk behaviors after treatment. The MITI is a reliable and valid measurement of treatment fidelity for motivational interviewing targeting sexual risk behaviors in people living with HIV/AIDS.     

Questions and reflections: the use of motivational interviewing microskills in a peer-led brief alcohol intervention for college students

The purpose of this study was to examine the association between peer facilitator adherence to motivational interviewing (MI) microskills and college student drinking behavior. First year students (N=67) took part in a Brief Alcohol Screening and Intervention for College Students (BASICS) led by peer facilitators trained in MI and BASICS. Participants were assessed pre- and 2 weeks post-intervention on contemplation to change, as well as, pre- and 3 months post- intervention on drinking quantity. Independent coders used the Motivational Interviewing Treatment Integrity scale (MITI, Moyers, Martin, Manuel, & Miller, 2003) to evaluate therapist MI adherence. Peer facilitators met beginning proficiency in MI on scores of empathy, the ratio of MI adherent behaviors to non-adherent behaviors and the ratio of open questions to total questions as defined by the MITI. Results indicated that a higher number of closed questions was related to less contemplation and a higher number of open questions was related to more contemplation post intervention. A higher number of simple reflections was associated with increased drinking at the 3 month assessment, however, complex reflections were found to attenuate the effect of simple reflections on changes in drinking. These findings highlight the importance of competent reflective listening skills and the need for continual training and supervision for peer facilitators.     

Reliability and validity of the borderline personality disorder severity index

The psychometric properties and validity of the Borderline Personality Disorder Severity Index (BPDSI), a semistructured interview assessing the frequency and severity of manifestations of Borderline Personality Disorder (BPD) during a circumscribed period, were investigated in two studies. In study 1, patients with BPD (n = 15), with other personality disorders (PD; n = 18), and without Axis II disorders (but with Axis I disorders; n = 10) assessed with the SCID were interviewed with the BPDSI (1-yr. version). Patients also filled out a number of questionnaires. A second rater judged taped BPDSI interviews. The BPDSI appeared to yield highly reliable (ICC = .93) and internally consistent (Cronbach's alpha = .85) scores. The BPDSI strongly discriminated BPD patients from other patients, and was not related to other Axis II pathology. Concurrent and construct validity was excellent. In study 2, a version of the BPDSI suitable for use in treatment outcome research was investigated (3-month version) in a sample of 64 BPD patients, 23 Cluster C PD patients, and 20 nonpsychiatric controls. Again, reliability coefficients were excellent (ICC = 0.97; Cronbach's alpha = 0.93), and validity indices were good. Clinical norms were also derived. In a sample of 28 BPD patients, the instrument detected improvement during 6 months of psychotherapy.     

Rating Scales for the Assessment of Empathic Communication in Medical Interviews (REM): Scale Development,Reliability, and Validity

Empathy has been shown to affect the quality of care by enhancing the physician–patient relationship and promoting effective communication. In this study, the newly developed Rating Scales for the Assessment of Empathic Communication in Medical Interviews (REM), designed to assess empathy and confrontation in physician–patient interactions, were subjected to preliminary psychometric testing. Particular attention was paid to face validity, reliability, sensitivity, and practicality. A total of 118 volunteers were asked to assess transcribed interactions between physicians and a standardized patient using the REM. In order to assess the convergent validity of the REM, the Motivational Interviewing Treatment Integrity Code (MITI) and the Behavior Change Counseling Index (BECCI) were used. Factor analysis identified two subscales, reflecting empathy and confrontation. Interrater reliability coefficients for items ranged from r = .82 to .97; Cronbach’s alphas for the two subscales were .89 and .88, respectively. The convergent validity was supported by substantial correlations between the REM scores and the MITI scores and by significant correlations between the REM scores and the BECCI score. The REM demonstrated good psychometric properties suggesting the rating scales might be useful in clinical practice, research endeavors, and medical education. Further research is necessary to assess the test-retest reliability as well as the predictive validity of this instrument.     

Development and reliability of a Brazilian Portuguese version of the MCMI-III

This paper describes the methodology used to translate the MCMI-III into Brazilian Portuguese and presents results from a reliability study conducted in Brazil with the translated instrument. The sample was comprised of 222 Brazilian college students. Test-retest correlations between first and second testings (over a period of 7 to 14 days) ranged from .70 to .93 across 27 scales. Cronbach's alpha coefficients ranged from .54 to .87. The procedures described in this paper represent preliminary steps in the development of a Brazilian Portuguese translation of the MCMI-III. Further studies are needed to obtain reliability data with a clinical sample, evaluate the diagnostic validity of the scales, and derive appropriate base rates for the Brazilian population.     

Measuring Mental Toughness in Sport: A Psychometric Examination of the Psychological Performance Inventory–A and Its Predecessor

The Rorschach Inkblot Method (RIM) is a performance-based personality assessment instrument used in both clinical and research settings worldwide. This investigation examines response-level, interrater reliability of U.S. graduate students enrolled in the same doctoral program and divided into two unique participant groups: 20 of those with previous Comprehensive System (CS) training background who are now trained in Rorschach Performance Assessment System (R-PAS; initial coders N = 11, blind coders N = 9), and 19 of those trained solely with R-PAS (initial coders N = 10, blind coders N = 9). Trainees’ coding was analyzed based on a sample of 50 protocols (half per group) representing a total of 1,189 responses. Blind coding was used among trainees and coding between the two investigated training groups was analyzed for each variable at the response level and expressed as percent agreement, Gwet's AC, and kappa values. Rates of agreement between the trainees were similar across both groups, and most of the concordance rates can be characterized as excellent. Overall, these results suggest that trainees may learn how to code R-PAS variables with similar reliability levels whether or not they had prior CS knowledge.     

Reliability of judgments of stuttering-related variables: The effect of language familiarity

Previous studies demonstrate mixed results and some methodological limitations regarding judges’ ability to reliably assess stuttering-related variables in an unfamiliar language. The present study examined intra- and inter-rater reliability for percent syllables stuttered (%SS), stuttering severity (SEV), syllables per minute (SPM), and speech naturalness (NAT) when English-speaking judges viewed speech samples in English and in a language with which they had no or minimal familiarity (Spanish). Over two time periods, 21 judges viewed eight videos of four bilingual persons who stutter. Data were analyzed for relative and absolute intra- and inter-rater reliability as well as for an effect of language on time period differences. Intra- and inter-rater relative reliability were good or excellent for all measures in both languages, with the exception of inter-rater relative reliability for NAT in both languages and %SS in Spanish. Intra-rater absolute reliability was acceptable in both languages for NAT and SEV and unacceptable in both for SPM and %SS. Inter-rater absolute reliability in both languages was unacceptable for all measures, even with judges with the same training. There was a clinically significant effect of language on %SS scores, but, despite a statistically significant effect of language for SPM and SEV, the differences were not clinically significant. Results indicate that reliability across and within languages varies by measure and is impacted by intra- vs. inter-rater reliability, relative vs. absolute reliability, and language familiarity. Modifications in training may be able to address some of the limitations found, particularly with regard to SPM and NAT.     

Psychological type and conceptions of empathy in experienced counsellors

Definitions of empathy vary widely. One explanation is that the different conceptions reflect personality differences. We investigated this idea with a sample of fifteen experienced counsellor, using the Myers-Briggs Type Indicator to measure aspects of personality and semistructured interviews to measure conceptions and reported practice of empathy. Counsellors who preferred sensing were found to make more references to empathy as a state and intuitive types to process. Those who preferred thinking and judging had a more active conception and reported practice, while feeling and perceiving types referred to a passive approach. Further relationships predicted by Myer-Briggs theory, between preference for thinking and cognitive empathy, and feeling and affective empathy, were close to statistical significance. Therefore, different psychological types do seem to define empathy differently. Implications for counselling theory, practice and training and education are discussed.     

Observations of families in structured interactions: Parenting therapists provide reliable ratings of mothers' parenting

The reliability of observations of parenting by parenting therapists was assessed. An important predictor of externalizing behavior in children is quality of parenting. Data were videotapes of structured interactions in families with a child age 8–12 years referred to the evidence based Parent Management Training Oregon (PMTO) treatment program for child behavior problems. The therapists had clinical PMTO training but no training in systematic observation. PMTO observational coders with specific coder training were included as a reference for the therapists. Five therapists and two coders observed videotapes of 10 families and performed global evaluations of mothers' parenting skills. They used the coder's impression measure used in PMTO research. Scores were analyzed in a generalizability theory framework for the two groups of observers separately. Both observer types reliably rank‐ordered the mothers and assessed the level of parenting skills. PMTO therapists without coder training provided reliable ratings of parenting constructs relevant to the clinical PMTO program in a manner comparable to that of the trained reference coders.     

Family interventions in psychosis: a scale to measure therapist adherence

The efficacy of family interventions in psychosis is well documented. UK and USA schizophrenia treatment guidelines advocate the practice of family interventions within routine clinical services. However, less attention has been paid to the study of treatment fidelity and the tools used in its assessment. This study reports the inter-rater reliability of a new scale: Family Intervention in Psychosis-Adherence Scale (FIPAS). This measure is designed to assess therapist adherence to the Kuipers et al . (2002) family intervention in psychosis treatment manual. Reliability ratings were based on a sample of thirteen audiotapes drawn from a randomized controlled trial of family intervention. The results indicated that the majority of items of the FIPAS had acceptable levels of inter-rater reliability. The findings are discussed in terms of their implications for the training and monitoring of the effectiveness of practitioners for family interventions in psychosis.     

Test-retest reliability of the Mirowsky-Ross 2 x 2 Index of the Sense of Control

This study investigated the short-term stability of the 1991 Mirowsky-Ross 2 x 2 Index of the Sense of Control. From an ongoing longitudinal study, 304 subjects were randomly selected for test-retest interviews occurring 1 to 4 days after their regularly scheduled first follow-up interview. Test-retest reliability was assessed at the item level using percent agreement and weighted kappa. At the scale score level, reliability was assessed with the intraclass correlation coefficient (ICC). ICCs were also calculated within categories of demographic, socioeconomic, psychosocial, and functional status characteristics. There was moderate to substantial item-level agreement (mean weighted kappa = 51; weighted kappa range = .38 to .66). At the scale score level there was substantial agreement (ICC = .71). No appreciable differences in ICC values were found in the demographic, socioeconomic, psychosocial, and functional comparisons of status characteristics. Thus, this sense of control measure has acceptable test-retest reliability and is appropriate for use in longitudinal research.     

Defining the Assessment and Treatment Competencies to Deliver Low-Intensity Cognitive Behavior Therapy: A Multi-Center Validation Study

Despite the vastly increased dissemination of the low-intensity (LI) version of cognitive behavior therapy (CBT) for the treatment of anxiety and depression, no valid and reliable indices of the LI-CBT clinical competencies currently exist. This research therefore sought to develop and evaluate two measures: the low-intensity assessment competency scale (LIAC) and the low-intensity treatment competency scale (LITC). Inductive and deductive methods were used to construct the competency scales and detailed rating manuals were prepared. Two studies were then completed. The first study used a quantitative, fully-crossed design and the second a multi-center, quantitative longitudinal design. In study one, novice, qualified, and expert LI-CBT practitioners rated an LI-CBT assessment session (using the LIAC) and an LI-CBT treatment session (using the LITC). Study two used the LIAC and LITC across four training sites to analyze the competencies of LI-CBT practitioners over time, across raters, and in relation to the actor/patients’ feedback concerning helpfulness, the alliance, and willingness to return. Both the LIAC and LITC were found to be single factor scales with good internal, test-retest reliability and reasonable inter-rater reliability. Both measures were sensitive to measuring change in clinical competence. The LIAC had good concurrent, criterion, discriminant, and predictive validity, while the LITC had good concurrent, criterion, and predictive validity, but limited discriminant validity. A score of 18 accurately delineated a minimum level of competence in LI-CBT assessment and treatment practice, with incompetent practice associated with patient disengagement. These observational ratings scales can contribute to the clinical governance of the burgeoning use of LI-CBT interventions for anxiety and depression in routine services and also in the methods of controlled studies.     

Motor learning without physical practice: The effects of combined action observation and motor imagery practice on cup-stacking speed

In this study we explored training effects for combined action observation and motor imagery (AO + MI) instructions on a complex cup-stacking task, without physical practice. Using a Graeco-Latin Square design, we randomly assigned twenty-six participants into four groups. This counterbalanced the within-participant factor of practice condition (AO + MI, AO, MI, Control) across four cup-stacking tasks, which varied in their complexity. On each of the three consecutive practice days participants experienced twenty trials under each of the three mental practice conditions. On each trial, a first-person perspective video depicted bilateral cup-stacking performed by an experienced model. During AO, participants passively observed this action, responding only to occasional colour cues. For AO + MI, participants imagined performing the observed action and synchronised their concurrent MI with the display. For MI, a sequence of pictures cued imagery of each stage of the task. Analyses revealed a significant main effect of practice condition both at the ’surprise’ post-test (Day 3) and at the one-week retention test. At both time points movement execution times were significantly shorter for AO + MI compared with AO, MI and the Control. Execution times were also shorter overall at the retention compared with the post-test. These results demonstrate that a complex novel motor task can be acquired without physical training. Practitioners can therefore use AO + MI practice to supplement physical practice and optimise skill learning.     

非正态分布测量数据对克隆巴赫信度α系数的影响

对“增大被试群体异质性,能提高测验信度”观点提出质疑。讨论非正态分布测量数据的偏度对克隆巴赫信度α系数的影响,以模拟方法验证Box—COX正态化变换对信度的高估现象,进而给出对克隆巴赫信度α系数估计的改进方法。