Construct Validity of ADHD/ODD Rating Scales: Recommendations for the Evaluation of Forthcoming DSM-V ADHD/ODD Scales

Exploratory structural equation modeling (SEM) was applied to a multiple indicator (26 individual symptom ratings) by multitrait (ADHD-IN, ADHD-HI and ODD factors) by multiple source (mothers, fathers and teachers) model to test the invariance, convergent and discriminant validity of the Child and Adolescent Disruptive Behavior Inventory with 872 Thai adolescents and the ADHD Rating Scale-IV and ODD scale of the Disruptive Behavior Inventory with 1,749 Spanish children. Most of the individual ADHD/ODD symptoms showed convergent and discriminant validity with the loadings and thresholds being invariant over mothers, fathers and teachers in both samples (the three latent factor means were higher for parents than teachers). The ADHD-IN, ADHD-HI and ODD latent factors demonstrated convergent and discriminant validity between mothers and fathers within the two samples. Convergent and discriminant validity between parents and teachers for the three factors was either absent (Thai sample) or only partial (Spanish sample). The application of exploratory SEM to a multiple indicator by multitrait by multisource model should prove useful for the evaluation of the construct validity of the forthcoming DSM-V ADHD/ODD rating scales.     

Construct Validity of ADHD-IN,ADHD-HI,ODD toward Adults,Academic and Social Competence Dimensions with Teacher Ratings of Thai Adolescents: Additional Validity for the Child and Adolescent Disruptive Behavior Inventory

Confirmatory factor analysis was used to test the construct validity of an attention-deficit/hyperactivity disorder-inattention, attention-deficit/hyperactivity disorder-hyperactivity/impulsivity, oppositional defiant disorder toward adults, academic and social competence factor model with teacher ratings of Thai adolescents (n = 872) with the Child and Adolescent Disruptive Behavior Inventory. The five-factor model resulted in an adequate fit in an absolute sense (i.e., CFI = .960; TLI = .985; RMSEA = .065; and WRMR = .883). All the items had significant and substantial loadings on their respective factors (i.e., > .78) with the five-factors showing discriminant validity. The five-factor model also resulted in similar results for boys and girls separately as well as younger and older adolescents. The current findings with the teacher version of the measure in conjunction with earlier research with mothers’ and fathers’ ratings of Brazilian, Thai, and American children (Burns et al., Psychological Assessment, 20, 121-130, 2008) and Thai adolescents (Burns et al., Psychological Assessment, 21, 635-641, 2009) provide increasing support for the construct validity of Child and Adolescent Disruptive Behavior Inventory within multiple cultures. Procedures are also outlined to improve the content validity as well as test the construct validity of forthcoming parent and teacher DSM-V ADHD/ODD rating scales.     

External Validity of the New Devereux Adolescent Behavior Rating Scales

The discriminant and concurrent validity of the five new scales for the Devereux Adolescent Behavior Rating Scale (DAB) was explored using a heterogeneous sample of psychiatric and substance abuse patients. Consistent with predictions, the substance abuse patients scored higher on the Acting Out Behaviors (AOB) and Heterosexual Interests (HI) scales, and psychiatric patients scored higher on the Psychotic Behaviors scale. Gender differences also were found, including boys being rated higher on Acting Out Behaviors, and girls higher on Heterosexual Interests. The new DAB scales demonstrated sufficient concurrent validity using a thorough record review and a parent rating scale (the Child Behavior Checklist [CBCL]). The Neurotic/Dependent Behaviors scale (NDB) showed a consistent relationship with substance abuse and several other measures of more externalizing behaviors, in addition to the predicted relationships with anxious, tense, and dependent behaviors. The Withdrawn/Timid Behaviors scale (WTB) proved to be a purer measure of internalizing behaviors in both sexes.     

ADHD and Depression Symptoms in Parent Couples Predict Response to Child ADHD and ODD Behavior

Parents of children with attention-deficit hyperactivity disorder (ADHD) and oppositional defiant disorder (ODD) often have elevated ADHD and depressive symptoms, both of which increase the risk of ineffective parenting and interparental discord. However, little is known about whether child ADHD/ODD behavior and parent ADHD or depressive symptoms uniquely or synergistically predict the quality of parenting and interparental communication during triadic (mother-father-child) interactions. Ninety parent couples, including 51 who have children diagnosed with ADHD, were randomly assigned to interact with a 9–12 year-old confederate child (84 % male) exhibiting either ADHD/ODD-like behavior or typical behavior. Parents reported their own ADHD and depressive symptoms, and parents and observers rated the quality of parenting and interparental communication during the interaction. Actor-partner interdependence modeling indicated that child ADHD/ODD behavior predicted less positive and more negative parenting and communication, independent of adult ADHD and depressive symptoms. Parent couples including two parents with elevated ADHD communicated more positively while managing children exhibiting ADHD/ODD behavior than couples managing children behaving typically or couples with only one parent with elevated ADHD symptoms. Couples including one parent with, and one parent without, elevated ADHD or depressive symptoms parented less positively and more negatively, and communicated more negatively, when managing children exhibiting ADHD/ODD behavior than when managing children behaving typically. Taken together, depending on the similarity of ADHD and depressive symptom levels in parent couples, adults managing children exhibiting ADHD/ODD behavior may parent or communicate positively or negatively. Findings highlight the need to consider the psychopathology of both parents when treating children with ADHD in two-parent homes.     

Maternal ADHD: Parent-Child Interactions and Relations With Child Disruptive Behavior

This study examined how ADHD symptoms in mothers of children with ADHD relate to their behavior during parent-child interactions and to their children's disruptive behavior. Findings indicated that mothers’ retrospective self-ratings of ADHD symptoms were related to their present negativity during parent-led play. Mothers’ self-ratings of current inattentive symptoms were related to their impatience during child-led play. Maternal ADHD symptoms were also related to their ratings of their children's ADHD and oppositional-defiant behaviors. Identifying relations between maternal psychopathology, such as ADHD symptomatology, and behaviors during parent-child interactions may yield clues to additional parent behavioral changes that would enhance treatment outcomes for young children with ADHD.     

Elucidating the Construct Validity of the Psychopathic Personality Inventory Triarchic Scales

This study sought to replicate and extend Hall and colleagues’ (2014) work on developing and validating scales from the Psychopathic Personality Inventory (PPI) to index the triarchic psychopathy constructs of boldness, meanness, and disinhibition. This study also extended Hall et al.'s initial findings by including the PPI Revised (PPI–R). A community sample (n = 240) weighted toward subclinical psychopathy traits and a male prison sample (n = 160) were used for this study. Results indicated that PPI–Boldness, PPI–Meanness, and PPI–Disinhibition converged with other psychopathy, personality, and behavioral criteria in ways conceptually expected from the perspective of the triarchic psychopathy model, including showing very strong convergent and discriminant validity with their Triarchic Psychopathy Measure counterparts. These findings further enhance the utility of the PPI and PPI–R in measuring these constructs.     

Concurrent Validity of the Child Behavior Checklist DSM-Oriented Scales: Correspondence with DSM Diagnoses and Comparison to Syndrome Scales

This study used receiver operating characteristic (ROC) methodology and discriminative analyses to examine the correspondence of the Child Behavior Checklist (CBCL) rationally-derived DSM-oriented scales and empirically-derived syndrome scales with clinical diagnoses in a clinic-referred sample of children and adolescents (N = 476). Although results demonstrated that the CBCL Anxiety, Affective, Attention Deficit/Hyperactivity, Oppositional and Conduct Problems DSM-oriented scales corresponded significantly with related clinical diagnoses derived from parent-based structured interviews, these DSM-oriented scales did not evidence significantly greater correspondence with clinical diagnoses than the syndrome scales in all cases but one. The DSM-oriented Anxiety Problems scale was the only scale that evidenced significantly greater correspondence with diagnoses above its syndrome scale counterpart —the Anxious/Depressed scale. The recently developed and rationally-derived DSM-oriented scales thus generally do not add incremental clinical utility above that already afforded by the syndrome scales with respect to corresponding with diagnoses. Implications of these findings are discussed.     

Establishing Norms on the Eyberg Child Behavior Inventory to Identify Disruptive Behaviors in Taiwanese Children

Abstract

Like other cultures, measures to accurately and sensitively assess and treat disruptive behaviors in Taiwanese children are necessary. This research provides normative and psychometric data (i.e., item-total correlations, split-half coefficients, internal consistency, and internal validity) on the Eyberg Child Behavior Inventory (ECBI), including clinical cut-off scores, with a Taiwanese community sample. The results suggest that the ECBI may be helpful in screening for disruptive behaviors in Taiwanese children, which may allow for appropriate prevention and intervention efforts. Although results are comparable to other research, cultural differences do exist, thereby highlighting the importance of cultural considerations in assessing child behavior, irrespective of country of treatment.     

Research Validity Scales for the NEO-PI-R: Additional Evidence for Reliability and Validity

We examined the reliability and validity of the research validity scales (Schinka, Kinder, & Kremer, 1997) for the NEO-Personality Inventory-Revised (NEO-PI-R) in a clinical sample. The Negative Presentation Management (NPM) and Positive Presentation Management (PPM) scales were found to have satisfactory internal consistency reliability. Support for the validity of these scales was provided by the pattern of convergent and discriminant correlations with respective Personality Assessment Inventory (PAI) validity scales. Finally, PAI profiles of individuals with invalid NPM scores were found to differ significantly from those with valid NPM scores. Comparisons of the invalid profiles with profiles from other clinical samples provided additional support for the use of the NPM scale as a measure of negative impression management.     

Item response theory analyses of adolescent self-ratings of the ADHD symptoms in the Disruptive Behavior Rating Scale

The graded response model (GRM) was used to evaluate the item response theory properties of the ADHD inattention and hyperactivity/impulsivity symptoms in the Disruptive Behavior Rating Scale-Self Report (DBRS-SR). This measure was completed by 363 adolescents, between 12 and 17 years of age. The findings showed that all symptoms were generally good for discriminating their respective latent traits. For most symptoms, their threshold values suggested that they were good at representing the appropriate traits from the mean trait levels, and their information values began to increase substantially from around −1 SD from the mean. These findings indicate good psychometric properties for the DBRS-SR. The practical and clinical implications of the findings are discussed.     

Preliminary Validity of the Eyberg Child Behavior Inventory with Taiwanese Clinic-Referred Children

The Eyberg Child Behavior Inventory (ECBI) has previously been shown to be a psychometrically sound instrument used to assess disruptive behaviors in children in the United States and in other cultures/countries but not in Taiwan. The purpose of this study was to examine the factor structure and to establish the discriminative validity of the ECBI with two groups of Taiwanese children: 70 clinic-referred children with clinically elevated externalizing behavior problems and 70 community-based matched comparison children. Exploratory factor analyses resulted in a six-factor model for the clinic-referred sample and a five-factStrengths and Difficultieor model for the matched comparison sample, indicating that the ECBI is not unidimensional. Adequate convergent and divergent validity also were established between the ECBI Intensity and Problem Scales and another measure of child externalizing (for assessing convergent validity) and internalizing (for determining divergent validity) behavior. The results of the present study suggest that the ECBI is a valid measure of assessing externalizing behavior problems in Taiwanese children. Future research may seek to refine the factor structure of the ECBI in a Taiwanese sample. Future studies are also needed to examine other psychometrics of the ECBI, replicate this study with a larger sample, and establish its normative data in Taiwan.     

Preliminary Validity of the Eyberg Child Behavior Inventory With Filipino Immigrant Parents

Filipinos are an understudied minority affected by significant behavioral health disparities. We evaluate evidence for the reliability, construct validity, and convergent validity of the Eyberg Child Behavior Inventory (ECBI) in 6- to 12-year-old Filipino children (N = 23). ECBI scores demonstrated high internal consistency, supporting a single-factor model (preintervention α = .91; postintervention α = .95). Results document convergent validity with the Child Behavior Checklist Externalizing scale at pretest (r = .54, p < .01) and posttest (r = .71, p < .001). We conclude that the ECBI is a promising tool to measure behavior problems in Filipino children.     

Factor Structure and Discriminative Validity of the Eyberg Child Behavior Inventory With Young Children

This study examined the factor structure and discriminative validity of the Eyberg Child Behavior Inventory (ECBI) with community-based and clinic-referred samples of young children (2–6 years). In the community sample, confirmatory factor analysis indicated a three-factor structure with components reflecting Inattentive, Oppositional Defiant, and Conduct Problem Behavior. In the clinic sample, component scores differentiated children with specific behavior problems from children without significant externalizing symptoms. However, component scores did not differentiate within the externalizing spectrum. The data suggest that ECBI components may be useful to screen community-based and clinic-referred young children for externalizing symptoms, but may be less useful to identify children with specific behavior problems.     

Assessing ADHD and comorbid disorders in children: the Child Behavior Checklist and the Devereux Scales of Mental Disorders

Evaluated discriminant validity and clinical utility of selected subscales of the Devereux Scales of Mental Disorders (DSMD; Naglieri, LeBuffe, & Pfeiffer, 1994) and the Child Behavior Checklist (CBCL; Achenbach, 1991a) in 228 children referred to a clinic for the evaluation and treatment of attention deficit hyperactivity disorder (ADHD). The DSMD is a multiaxial behavior rating scale that measures symptomatology for a broad range of child psychopathology as described in the Diagnostic and Statistical Manual of Mental Disorders (3rd ed., rev. [DSM-R-III] and 4th ed. [DSM-IV]; American Psychiatric Association, 1987, 1994). Discriminant function analyses as well as sensitivity, specificity, and predictive power analyses were computed to evaluate the discriminant validity and clinical utility of selected DSMD and CBCL subscales for assessing ADHD, oppositional defiant disorder (ODD), and anxiety disorders. Results indicated that the DSMD compared very favorably with the CBCL in the ability to discriminate between children with ADHD and those without ADHD and between children with comorbid ODD and anxiety disorders and children who did not meet criteria for these disorders. The DSMD Attention subscale may be somewhat better at ruling in ADHD combined subtype (ADHD-C) and ADHD inattentive subtype (ADHD-I) than the CBCL Attention Problems subscale, but the CBCL Attention Problems subscale may have slightly better utility than the DSMD Attention subscale in ruling out these subtypes. Both the CBCL and DSMD were more useful for ruling out than for ruling in ODD and anxiety disorders.     

Concurrent Validity of the Personality Assessment Inventory Borderline Scales in Patients Seeking Dialectical Behavior Therapy

Drawing on a large, nationally representative sample of young adults (the National Longitudinal Study of Adolescent Health; N = 15,701; M age = 29.10), we evaluated the psychometric properties of the Mini-IPIP, a 20-item inventory designed to concisely assess the 5 factors of personality. The results suggest that the Mini-IPIP has a 5-factor structure; most of the scales have acceptable reliability; all the scales have partial or full metric invariance; and the scales exhibit some degree of criterion validity. However, the absence of scalar invariance for many of the scales suggests caution when comparing personality scores among groups defined by sex or race and ethnicity. We offer practical considerations for researchers interested in using this inventory with this sample, and also suggestions for modification of the Mini-IPIP.     

Assessing Stress in Families of Children with ADHD: Preliminary Development of the Disruptive Behavior Stress Inventory (DBSI)

This paper describes the initial development of the Disruptive Behavior Stress Inventory (DBSI). This 40-item measure, which provides indices of the number of stressors experienced during the past 6 months and the rated stressfulness of these events, is designed to assess behavior-related family stress experienced by families of children with attention-deficit/hyperactivity disorder (ADHD). Findings provide strong support for the internal consistency of the DBSI and suggest adequate test-retest reliability. Support for the validity of the measure is suggested by findings that DBSI stress indices differentiate between parents of children with ADHD and those with no history of this disorder. Additional support is provided by findings that scores on the DBSI also differentiate between parents of children with ADHD, combined type and parents of children with ADHD, inattentive type. It is suggested that this measure may provide useful clinical information regarding stress levels in families of children with ADHD and that it may be of potential value as an ADHD treatment outcome measure.     

A Psychometric Analysis of the Child Behavior Checklist DSM-Oriented Scales

The Child Behavior Checklist for Ages 6–18 (CBCL/6-18) possesses newly developed DSM-Oriented Scales, constructed through expert clinical judgment to match selected categories for behavioral/emotional problems as described in the DSM-IV. The present investigation examined the basic psychometric properties for all six DSM-Oriented Scales (i.e., Affective, Anxiety, Somatic, Attention-Deficit/Hyperactivity, Oppositional, and Conduct Scales) in a large clinical sample of children and adolescents (N = 673). Findings from the present study provide strong evidence for the reliability, as well as convergent and discriminative validity, of these scales. It appears that the DSM-Oriented Scales may provide accurate supplementary information that may be considered when formulating clinical diagnoses.

Mediators and Moderators of the Relation between Parental ADHD Symptomatology and the Early Development of Child ADHD and ODD Symptoms

The present study examined mediators and moderators of the relation between parental ADHD symptomatology and the development of child attention-deficit/hyperactivity disorder (ADHD) and oppositional defiant disorder (ODD) symptoms across the preschool years. Participants included 258 (138 boys) 3-year-old children (M = 44.13 months, SD = 3.39) with and without behavior problems and their parents who took part in a 3-year longitudinal study. Maternal ADHD symptoms predicted later ADHD symptoms in children, controlling for early child symptomatology. Both family history of ADHD and paternal comorbid psychopathology predicted later child ADHD and ODD symptoms, but they did not account for the association between maternal and child ADHD symptoms. Although paternal ADHD symptoms were associated with age 3 child ADHD symptoms, they did not significantly predict later child ADHD symptoms controlling for early symptomatology. Family adversity moderated the relation between maternal ADHD and child ADHD symptoms, such that the relation between maternal and child ADHD symptoms was stronger for families with less adversity. Maternal overreactive parenting mediated the relation between maternal ADHD symptoms and later child ADHD and ODD symptoms. Our findings suggest that targeting paternal comorbid psychopathology and maternal parenting holds promise for attenuating the effects of parental ADHD on children’s ADHD.     

Construct Validation of Scales Derived From the Rorschach Method: A Review of issues and Introduction to the Rorschach Rating Scale

I suggest the main goal of Rorschach validation should be a refined understanding of what each score means. Toward this end, I review general issues in construct validity, hurdles unique to the Rorschach, and general limitations with validation criteria. I then recommend two approaches for improving criteria so they can begin approximating the gold standards that are necessary for a refined understanding of what scores actually measure. The first is a method for improving expert clinical judgment, and the second is a method for aggregating data across diverse judges. Finally, the Rorschach Rating Scale (RRS) is presented as a criterion tool to be used with either of these approaches to validation. The RRS is a fairly comprehensive summary of the constructs thought to be measured by various Rorschach scoring systems. The utility of the RRS for research and training are discussed, as are other practical, theoretical, and psychometric issues in its application.