Reliability of diagnosis in older psychiatric patients using the structured clinical interview for DSM-III-R

We conducted one of the few studies that has examined the reliability of the Structured Clinical Interview for DSM-III-R Axis I (SCID-I) with a mixed inpatient and outpatient population of adults 55 years old and over (range, 56–84 years; mean, 67.33 years). All SCID interviews were videotaped or audiotaped and were administered by Master's-level clinicians working toward their doctorate degrees in clinical psychology. Interrater reliability estimates (kappa and percentage agreement) were calculated for current major depressive episode (47% base rate) and the broad diagnostic categories of anxiety disorders (15% base rate) and somatoform disorders (12% base rate). Kappa values were .70, .77, and 1.0. Respective percentage agreement was 85% for major depression, 94% for anxiety disorders, and 100% for somatoform disorders. Overall percentage agreement was 91%. We conclude that the SCID-I can be effectively administered by relatively inexperienced clinicians to diagnose older psychiatric patients reliably. Directions that future research might take are offered.     

Simple Procedures to Estimate Chance Agreement and Kappa for the Interrater Reliability of Response Segments Using the Rorschach Comprehensive System

When determining interrater reliability for scoring the Rorschach Comprehensive System (Exner, 1993), researchers often report coding agreement for response segments (i.e., Location, Developmental Quality, Determinants, etc.). Currently, however, it is difficult to calculate kappa coefficients for these segments because it is tedious to generate the chance agreement rates required for kappa computations. This study facilitated kappa calculations for response segments by developing and validating formulas to estimate chance agreement. Formulas were developed for 11 segments using 400 samples, cross-validated on 100 samples, and applied to the data from 5 reliability studies. On cross-validation, the validity of the prediction formulas ranged from .93 to 1.0 (M = .98). In the 5 reliability studies, the average difference between estimated and actual chance agreement rates was .00048 and the average difference between estimated and actual kappa values was .00011 (maximum = .0052). Thus, the regression formulas quite accurately predicted chance agreement rates and kappa coefficients for response segments.     

A practical method for screening psychiatric disorder in children with speech and language disorders

Children with speech and language disorders have been found to have a prevalence rate of psychiatric disorder of 50%. The authors sought to develop an efficient screening procedure to distinguish such children who have psychiatric disorder and those who do not. Cutoff scores for easily administered parent and teacher behavior rating questionnaires were investigated in a population of 256 children with speech and language disorders and known presence or absence of psychiatric disorder. It was learned that when the questionnaires are used together, a cutoff score of 31 or greater on either questionnaire has a positive predictive value of 74.1 % and a negative predictive value of 76.1%. This screening method shows promise in assisting speech pathologists in comprehensive evaluation and treatment planning for children with speech and language disorders.     

Auditory temporal pattern learning in children with speech and language impairments

Four children demonstrating speech and language impairments were examined with respect to their ability to learn to identify certain auditory temporal perceptual information. These children listened to six-element temporal patterns and made judgments about the temporal proximity of two of the elements. Subjects listened to the patterns over a number of exposures ranging from 6 to 14, depending on the subject. Performance on the task improved significantly with repeated exposures. However, the disordered subjects' best performance was still significantly poorer than normal children who had only 1 exposure to the task. These results suggest that, in part, performance differences on temporal perceptual tasks between speech and language disordered children and normal children can be accounted for by differences in perceptual learning. However, because the disordered children never reached normal levels, learning differences may be associated with a fundamental deficit in temporal processing or some other mechanism such as impaired attention.     

Short-interval test-retest interrater reliability of the Dutch version of the Structured Clinical Interview for DSM-IV personality disorders (SCID-II)

This study examined the short-interval test-retest reliability of the Structured Clinical Interview (SCID-II: First, Spitzer, Gibbon, & Williams, 1995) for DSM-IV personality disorders (PDs). The SCID-II was administered to 69 in- and outpatients on two occasions separated by 1 to 6 weeks. The interviews were conducted at three sites by ten raters. Each rater acted as first and as second rater and equal number of times. The test-retest interrater reliability for the presence or absence of any PD was fair to good (kappa = .63) and was higher than values found in previous short-interval test-retest studies with the SCID-II for DSM-III-R. Test-retest reliability coefficients for trait and sumscores were sufficient, except for dependent PD. Values for single criteria were variable, ranging from poor to good agreement. Further large-scale test-retest research is needed to test the interrater reliability of more categorical diagnoses and single traits.     

Psychiatric and substance use disorders as risk factors for attempted suicide among adolescents: a case control study

The objective of this research was to test substance-related and non-substance-related psychiatric disorders as predictors of attempted suicide among adolescents. Ninety-six psychiatrically disordered suicide attempters were matched one-to-one to 96 psychiatrically disordered non-attempters on age, race, gender, and the presence/absence of major depression. Conditional logistic regression was used to test psychiatric risk factors for their power to predict attempted suicide among adolescents. Bipolar disorder, cocaine use disorders, and conduct disorder were found to be predictive of attempted suicide in univariate testing. Bipolar disorder, inhalant use disorders, cocaine use disorders, and hallucinogen use disorders were found to be predictive of attempted suicide, after adjusting for all other covariates. Loglinear analyses revealed high odds ratios associated with the comorbidities of alcohol use disorder with conduct disorder and drug use disorders with conduct disorder in both groups. Higher rates of cocaine use disorder/conduct disorder, hallucinogen use disorder/conduct disorder, and alcohol use disorder/ conduct disorder were found among suicide attempters. Evaluation of these particular comorbid conditions should be part of the adolescent suicide risk assessment.     

Primary Process Content in Paranoid Schizophrenic Speech

Free speech samples given by paranoid schizophrenics, nonschizophrenic paranoids, and nonpsychotic psychiatric patients were submitted to computerized content analysis. Speech samples were searched for words belonging to the Regressive Imagery Dictionary (Martindale, 1975), which yields a well-validated measure of primary process content. Three word-concreteness dictionaries were also employed. Compared to the other groups, paranoid schizophrenics produced speech higher in primary process content as well as in transitive verb concreteness. Results are consistent with psychoanalytic theory.     

Primary process content in paranoid schizophrenic speech

Free speech samples given by paranoid schizophrenics, nonschizophrenic paranoids, and nonpsychotic psychiatric patients were submitted to computerized content analysis. Speech samples were searched for words belonging to the Regressive Imagery Dictionary (Martindale, 1975), which yields a well-validated measure of primary process content. Three word-concreteness dictionaries were also employed. Compared to the other groups, paranoid schizophrenics produced speech higher in primary process content as well as in transitive verb concreteness. Results are consistent with psychoanalytic theory.     

Interrater reliability of the DSM-III-R with preschool children

Little attention has been paid to evaluating the use of DSM-III-R with preschool children. Children (N = 510) ages 2 to 5 years who were screened at the time of a pediatric visit were selected to participate in an evaluation which included questionnaires, a semistructured interview, developmental testing, and a play observation. Following the evaluation, two clinical child psychologists independently assigned DSM-III-R diagnoses. For each diagnostic category, kappa and Ycoefficients were calculated; Ycoefficients are less sensitive to base rates of disorders. For overall agreement, the weighted mean kappa (.61), and mean Y(.66) were moderately high. Overall agreement that the child had at least one of the disruptive disorders was substantial (kappa =.64; Y =.65);agreement that there was at least one of the emotional disorders was moderate for kappa (.54), but substantial for Y(.70). Kappa coefficients were higher for major categories of disorder than for specific disorders; however, Ycoefficients did not show a decline for specific disorders. Interrater reliability of DSM-III-R appears to be similar for preschoolers and older children.This study was supported by grant MH46089 from the National Institute of Mental Health.A preliminary report was presented at the Fifth Annual NIMH International Research Conference on the Classification and Treatment of Mental Disorders in General Medical Settings, Bethesda, Maryland, September 1991. We gratefully acknowledge the members of the Pediatric Practice Research Group who participated in this study.     

A review of the observational data-collection and reliability procedures reported in The Journal of Applied Behavior Analysis

The research published in the Journal of Applied Behavior Analysis (1968 to 1975) was surveyed for three basic elements: data-collection methods, reliability procedures, and reliability scores. Three-quarters of the studies reported observational data. Most of these studies' observational methods were variations of event recording, trial scoring, interval recording, or time-sample recording. Almost all studies reported assessment of observer reliability, usually total or point-by-point percentage agreement scores. About half the agreement scores were consistently above 90%. Less than one-quarter of the studies reported that reliability was assessed at least once per condition.     

Rorschach interrater agreement estimates: An empirical evaluation

A standardized estimation of Rorschach interrater agreement is needed. Percentage agreement, although widely used, is found to be unsuitable. Forty-one protocols from adults in both a normal and a psychiatric sample were scored by two or three scorers, making 85 scoring pairs. Percentage agreement, correlations (phi and Pearson's r ), and kappa were computed on single response, total score, and category level. Percentage agreement shows minimal variation. Even when exceeding 0.80, it can obscure major disagreements. Kappa and correlations both vary in a similar way with level of disagreement. Total score level does not give additional information compared to single score and category levels. Kappa proved to be conservative and reliable and is therefore suggested as a standard estimate.     

Patterns of adaptation and personality disorders

Patterns of adaptation to conflict were explored with the Serial Color-Word Test, and personality disorders were assessed by means of the Coolidge Axis II Inventory in a group of 76 nonpsychotic women volunteers in the age range 18-50 yr. (M=29.1 yr., SD=8.3), who attended a psychiatric outpatients unit. Forward multiple regression analyses were performed to investigate whether patterns of adaptation were associated with personality disorders. 10 out of 13 personality scales, as measured by the Coolidge Axis II Inventory, were significantly predicted by adaptive variables. Some predictors were positive and others were negative. The variable R(AD) was a negative predictor of avoidant and dependent personalities, and a positive predictor of Extraversion, Aggressive personality, and Antisocial personality; this finding suggests that R(AD) may represent the regulative counterpart of a continuum from passive introversion to aggressive extraversion. The results encourage further research on nontrait laboratory correlates of personality disorders.     

Multidisciplinary assessment and diagnosis of conversion disorder in a patient with foreign accent syndrome

Multiple reports have described patients with disordered articulation and prosody, often following acute aphasia, dysarthria, or apraxia of speech, which results in the perception by listeners of a foreign-like accent. These features led to the term foreign accent syndrome (FAS), a speech disorder with perceptual features that suggest an indistinct, non-native speaking accent. Also correctly known as psuedoforeign accent, the speech does not typically match a specific foreign accent, but is rather a constellation of speech features that result in the perception of a foreign accent by listeners. The primary etiologies of FAS are cerebrovascular accidents or traumatic brain injuries which affect cortical and subcortical regions critical to expressive speech and language production. Far fewer cases of FAS associated with psychiatric conditions have been reported. We will present the clinical history, neurological examination, neuropsychological assessment, cognitive-behavioral and biofeedback assessments, and motor speech examination of a patient with FAS without a known vascular, traumatic, or infectious precipitant. Repeated multidisciplinary examinations of this patient provided convergent evidence in support of FAS secondary to conversion disorder. We discuss these findings and their implications for evaluation and treatment of rare neurological and psychiatric conditions.     

The Collaborative Longitudinal Personality Disorders Study: reliability of axis I and II diagnoses

Both the interrater and test-retest-retest reliability of axis I and axis II disorders were assessed using the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I) and the Diagnostic Interview for DSM-IV Personality Disorders (DIPD-IV). Fair-good median interrater kappa (.40-.75) were found for all axis II disorders diagnosed five times or more, except antisocial personality disorder (1.0). All of the test-retest kappa for axis II disorders, except for narcissistic personality disorder (1.0) and paranoid personality disorder (.39), were also found to be fair-good. Interrater and test-retest dimensional reliability figures for axis II were generally higher than those for their categorical counterparts; most were in the excellent range (> .75). In terms of axis I, excellent median interrater kappa were found for six of the 10 disorders diagnosed five times or more, whereas fair-good median interrater kappa were found for the other four axis I disorders. In general, test-retest reliability figures for axis I disorders were somewhat lower than the interrater reliability figures. Three test-retest kappa were in the excellent range, six were in the fair-good range, and one (for dysthymia) was in the poor range (.35). Taken together, the results of this study suggest that both axis I and axis II disorders can be diagnosed reliably when using appropriate semistructured interviews. They also suggest that the reliability of axis II disorders is roughly equivalent to that reliability found for most axis I disorders.     

Development of a Structured Psychiatric Interview for Children: Agreement Between Child and Parent on Individual Symptoms

To test the reliability of children's reporting as compared with that of their mothers, a highly structured psychiatric diagnostic interview was used with 307 subjects, ages 6 through 16. Another interviewer gave each mother a similar interview about the child. Responses of each mother-child pair to 168 questions were compared using the kappa statistic. Highest agreement was found on questions concerning symptoms that are concrete, observable, severe, and unambiguous. Mothers tended to report significantly more behavioral symptoms, and children more subjective symptoms. Reasons for low kappas and asymmetrical reporting of symptoms are discussed.     

A comparison of self-report and interview diagnoses of DSM-III-R personality disorders

A total of 73 psychiatric inpatients, all of whom (but two) fulfilled criteria for at least one specific personality disorder (PD) on SCID-II PQ, were interviewed with the help of PDE. The self-report PD diagnosis was confirmed in 35 (48 per cent) patients. The diagnostic agreement between the two instruments was poor, yielding an overall weighted kappa of 0.22. Levelling off the PD base rates by increasing or decreasing the diagnostic threshold of SCID-II PQ and PDE respectively increased the overall weighted kappa to 0.38 in both instances. 70 per cent of SCID-II PQ but only 29 per cent of PDE personality disorders were of extensive type. Most frequent important co-occurrences occurred between individual PD types within cluster 2. On the whole, the results confirmed the relatively poor agreement between self-report and interview PD diagnoses. The utilization of self-report questionnaires in a clinical practice remains a controversial issue. © 1998 John Wiley & Sons, Ltd.     

Development and validation of the Eating Disorder Diagnostic Scale: a brief self-report measure of anorexia, bulimia, and binge-eating disorder

This article describes the development and validation of a brief self-report scale for diagnosing anorexia nervosa, bulimia nervosa, and binge-eating disorder. Study 1 used a panel of eating-disorder experts and provided evidence for the content validity of this scale. Study 2 used data from female participants with and without eating disorders (N = 367) and suggested that the diagnoses from this scale possessed temporal reliability (mean kappa = .80) and criterion validity (with interview diagnoses; mean kappa = .83). In support of convergent validity, individuals with eating disorders identified by this scale showed elevations on validated measures of eating disturbances. The overall symptom composite also showed test-retest reliability (r = .87), internal consistency (mean alpha = .89), and convergent validity with extant eating-pathology scales. Results implied that this scale was reliable and valid in this investigation and that it may be useful for clinical and research applications.     

Effects of speech rate on the absolute and relative timing of apraxic and conduction aphasic sentence production

The purpose of this investigation was to provide a constructive replication of the Kent and McNeil (1987, In Phonetic approaches to speech production in aphasia and related disorders. San Diego: College-Hill Press) study of the speech timing characteristics of apraxic and conduction aphasic speakers. Acoustic analysis was used to obtain absolute utterance durations, segment durations, and vowel formant trajectories from utterances produced under control, fast, and slow rate conditions. Segment-to-whole ratios and slope values were calculated. Results support the hypothesis presented by Kent and McNeil (1987) that there is a phonetic-motoric component contributing to the speech patterns of both the apraxic and conduction aphasic speakers sampled. Theories of rate control in normal and disordered speakers are discussed.     

Development of a structured psychiatric interview for children: Agreement between child and parent on individual symptoms

To test the reliability of children's reporting as compared with that of their mothers, a highly structured psychiatric diagnostic interview was used with 307 subjects, ages 6 through 16. Another interviewer gave each mother a similar interview about the child. Responses of each mother-child pair to 168 questions were compared using the kappa statistic. Highest agreement was found on questions concerning symptoms that are concrete, observable, severe, and unambiguous. Mothers tended to report significantly more behavioral symptoms, and children more subjective symptoms. Reasons for low kappas and asymmetrical reporting of symptoms are discussed.     

Symposium: Research In Projective Techniques

Psychotic and nonpsychotic psychiatric inpatients were tested on the Rorschach. "Reality-testing" was measured by four form-level scoring systems designed by Beck, Mayman, Becket, and Wilensky, The psychotic subjects were dichotomized on three diagnostic dimensions: schizophrenia, paranoia, and premorbid social adjustment. No significant differences were found for the Beck system. With the other systems, psychotics showed significantly poorer reality testing than nonpsychotics. No differenced were found for the schizophrenic and paranoid dimensions. The Phillips premorbid adjustment score was negatively correlated with all four measures of "reality-testing." The conclusion drawn was that Rorschach measures of "reality-testing" are associated with psychotic and poor premorbid functioning, but do not differentiate schizophrenic from nonschizophrenic psychotics or paranoid schizophrenics from nonparanoid schizophrenics.