DSM-IV: empirical guidelines from psychometrics

This commentary addresses the use of psychometric theory and methodology in the development of the 4th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). Reliability issues include interdiagnostician reliability, temporally consistent diagnoses, and the relations of diagnostic criteria within categories. Validity issues include content validity of the diagnostic criteria, criterion-related validity (the relation between different criterion sets or their algorithms and alternative diagnostic criteria), and construct validity (the relation between diagnostic categories and external validators). Specific questions and methodology to investigate its utility vary with the different uses proposed for the diagnostic system. Specific psychometric methodologies that may be useful in developing the DSM-IV are noted, as are the limitations of psychometrics and their applicability to DSM-IV.     

The Quality of the Childrearing Environment of Refugee or Asylum‐Seeking Children and the Best Interests of the Child: Reliability and Validity of the BIC‐Q

The Best Interest of the Child Questionnaire (BIC‐Q) has been designed as an instrument for screening the quality of the rearing situation of asylum‐seeking or refugee children. It is intended to aid legal decisions in asylum procedures. The aim of this study was to determine the reliability and the construct validity of the BIC‐Q. Based on a study sample of refugee or asylum‐seeking children in the Netherlands (N = 74), the psychometric quality of the BIC‐Q was investigated using Cohen's kappa for the inter‐ and intrarater reliability and a nonparametric item response model for the construct validity. The interrater and intrarater reliabilities of the BIC‐Q were good (kappa = .65 and .74 respectively). The results of the item response model revealed that the 14 pedagogical environmental conditions formed a strong and valid measurement scale for the quality of the childrearing environment (H = .55; rho = .94). Preliminary results indicate that the BIC‐Q may be applied to support decisions on where the asylum‐seeking or refugee child has the best opportunities for development. Copyright © 2012 John Wiley & Sons, Ltd.     

Clinical utility of dimensional models for personality pathology

Clinical utility is defined as the extent to which the DSM assists clinical decision makers in fulfilling the various clinical functions of a psychiatric classification system. Distinction is made between elements of diagnostic validity that are more or less conditional to utility (i.e., coverage, and consistency with etiology and prognosis) and components of clinical utility in the narrow sense of the term (i.e., user acceptability and accuracy, communication, reliability, subtlety, and clinical decision making). Clinical utility is often considered the driving force behind the respective revisions of the DSM system, yet it has been difficult if not impossible to obtain reasonable levels of clinical utility within categorical classifications of personality disorders. This paper presents evidence that a dimensional diagnostic system will substantially improve clinical utility, especially with respect to coverage, reliability, subtlety, and clinical decision making. In addition, some evidence suggests that the purely dimensional models outperform the so-called hybrid models (i.e., dimensional profiling of categories) with respect to coverage, consistency with etiology, subtlety, and clinical decision making. Several research options that might inform future revisions, including the choice or development of a dimensional classification, are suggested.     

Mutual Help Groups, Perceived Status Benefits, and Well-Being: A Test with Adult Children of Alcoholics with Personal Substance Abuse Problems

A field experiment was conducted to examine the potential effects of mutual help group participation on perceived status benefits, depression, and substance use among adult children of alcoholics (ACOAs) with personal substance abuse problems. Participants were randomly assigned to attend either ACOA-specific mutual help group meetings or substance abuse education classes during the initial month that they were enrolled in a residential treatment program for low-income substance abusers. Analyses were based on assessments made at baseline, a 1-month posttest, and a 6-month follow-up. Results indicated that participation in the mutual help group promoted perceived status benefits, which in turn led to reductions in depression and substance use. These findings have clear implications for consumers of ACOA groups and can inform research on the social psychological underpinnings of different types of mutual help groups.     

Two Is Better Than One

ABSTRACT— Psychologists too often rely on only one source of evidence to affirm the validity of a construct. However, they usually do not know all the conditions that can produce the evidence they gather. Hence, the inference is often limited to the data gathered and does not generalize to other categories of information. Examples of this habit are presented along with the suggestion that all social scientists should affirm the utility of their concepts with more than one class of information.     

The Influence of Posttreatment Mutual Help Group Participation on the Friendship Networks of Substance Abuse Patients

The effect of 12-step mutual help groups (e.g., Narcotics Anonymous) on members' friendship networks has received little attention. This 1-year longitudinal study examined such effects in a sample of 2,337 male substance abuse inpatients, 57.7% of whom became significantly involved in 12-step activities (e.g., reading program literature, attending meetings) after treatment. An a priori model of the interplay of 12-step involvement and friendship networks was tested using structural equation modeling, and found to have excellent fit to the data. Twelve-step group involvement after treatment predicted better general friendship characteristics (e.g., number of close friends) and substance abuse-specific friendship characteristics (e.g., proportion of friends who abstain from drugs and alcohol) at follow-up. Results are discussed in terms of how mutual help group involvement benefits patients and how the self-help group evaluation paradigm should be broadened.     

Reconceptualizing personality pathology in DSM-5: limitations in evidence for eliminating dependent personality disorder and other DSM-IV syndromes

The DSM-5 Personality and Personality Disorders Workgroup proposed that five DSM-IV personality disorders be eliminated as formal diagnostic categories (paranoid, schizoid, histrionic, narcissistic, and dependent), because these syndromes purportedly have low clinical utility and minimal evidence for validity. Scrutiny of studies cited in support of this proposal reveals difficulties in three areas: (1) Inadequate information regarding parameters of the literature search; (2) Mixed empirical support for proposed changes; and (3) Selective attention to certain disorders and not others. Review of validity and clinical utility data related to dependent personality disorder indicates that evidence regarding this syndrome does not differ from that of syndromes proposed for retention in DSM-5. Limitations in the research base cited by the workgroup illuminates gaps in the personality disorder literature, and may serve as a starting point for systematic research on personality pathology so that adequate empirical data are available to decide which syndromes to retain, revise, or remove in future versions of the diagnostic manual.     

Communicating the validity of structured job interviews with graphical visual aids

The structured job interview is often resisted by human resource (HR) professionals despite its superior predictive validity compared to the traditional unstructured interview. However, HR professionals may underestimate the utility of structured interviews because of how validity information is presented. Three online experiments with MTurk samples were conducted to examine the effect of statistical visual aids (Binomial effect size display; expectancy chart; icon array) on people's perceived usefulness of validity information and intentions toward using the structured interview. Results showed that graphical visual aids (expectancy chart and icon array) were perceived as more useful for communicating validity evidence than a nongraphical effect size display. People also judged the structured interview as more useful and were more willing to use it when validity evidence was presented graphically. Individual differences in graph literacy and nationality were also examined.     

College and Career Readiness Counseling Support Scales

This study presents initial evidence supporting the factor structure and criterion validity of the College and Career Readiness Counseling Support scales. Graduating 12th graders' ratings of both the frequency and helpfulness of meetings with counselors were correlated with malleable factors related to achievement in high school and with important markers of postsecondary success approximately 18 months after students graduated from high school. The scales can be used to help students self‐advocate and to evaluate counseling services.     

EXAMINING SELECTION UTILITY WHERE COMPETING PREDICTORS DIFFER IN ADVERSE IMPACT

This paper discusses the roles of validity, cut score choice, and adverse impact on selection system utility using data from two concurrent validation studies. We contrast an assessment center and published aptitude test on several metrics, including validity, testing costs, adverse impact, and utility. The assessment center produced slightly lower validity than the aptitude test while costing roughly 10 times as much per candidate. In spite of these advantages for the aptitude test, the assessment center produced so much less adverse impact its operational utility would be higher given cut scores likely to be chosen in this organization. Potential concerns with applying net utility models to this type of situation are discussed in comparison to gross utility models.     

EXPLORING OPTIONS FOR SUPPORTING TEST USE IN SITUATIONS PRECLUDING LOCAL VALIDATION

This paper examines some of the options available to practitioners interested in supporting the use of selection measures in an organization, including test transportability, validity generalization (VG), and synthetic validation, reviewing some of the advantages, disadvantages and requirements of each approach. Results of four unpublished, proprietary validation studies are reported which compare validity estimates provided by the job component validation (JCV; a type of synthetic validation) routine inherent in the Position Analysis Questionnaire (PAQ) with observed validity coefficients for a variety of criterion measures. We then examine the accuracy of the JCV procedure in predicting validity coefficients for 51 clerical classifications extracted from an existing PAQ job evaluation database of a large utility company. Predicted JCVs are compared to mean observed validity coefficients for five DOT clerical categories provided by Pearlman, Schmidt, and Hunter (1980). The VG and JCV methods provided highly similar and converging estimates of the validity of cognitive ability tests for predicting performance in clerical occupations. Implications for practice are discussed, particularly the need to use multiple, converging lines of evidence to support test use.     

Construct validity of the Rosenzweig Picture-Frustration Study

Projective and semiprojective techniques are both tools and tests. A schema for assessing the validity of projective devices is proposed: (1) Primary, consisting of (a) construct validity i.e., the soundness of the techniques vis-a-vis its underlying concepts; and (b) criterion-related validity, i.e., the concordance between tool results and specified external criteria. (2) Secondary, viz., pragmatic validity, i.e., the usefulness of the instrument in applied fields. Discussion then focuses on the construct validity of the conceptually-based, semiprojective Rosenzweig P-F Study. specifically the evidence on (a) the self-consistency of its conceptual dimensions; (b) the sequential. social development of the Types and Directions of Aggression; (c) levels of behavior elicited; (d) sensitivity to scorable changes after induced frustration: and (e) physiological correlates oft he aggression categories. In each of these areas evidence exists in support of the validity of the P-F Study.     

Clarifying a Construct: An Integrative Functional Model of Reassurance-Seeking Behaviors

Reassurance seeking (RS) has emerged as an important transdiagnostic psychological construct. However, RS has been conceptualized inconsistently and applied to functionally disparate behaviors within research and practice. In order to clarify the construct, the presently proposed Integrative Functional Model of Reassurance Seeking (IFM-RS) operationalizes and categorizes RS behaviors according to their functions. The following three categories comprise the model: Avoidant (negatively reinforced) RS, Evocative (positively reinforced) RS, and Bifunctional (simultaneously reinforced) RS. To evidence the validity and utility of this model, a review of relevant research is presented. The IFM-RS is then applied in conceptualizing the deleterious effects of RS on mental health and relevant treatment implications are subsequently discussed. We contend that adoption of the IFM-RS and its terminology by clinicians and researchers would facilitate accurate communication about the construct, improve the validity and applicability of RS research, and guide clinicians to effectively identify and target RS behaviors in the context of psychopathology.     

Assessing Sexual Orientation Symptoms in Obsessive-Compulsive Disorder: Development and Validation of the Sexual Orientation Obsessions and Reactions Test (SORT)

Obsessive-compulsive disorder (OCD) includes many symptom presentations, which creates unique diagnostic challenges. Fears surrounding one’s sexual orientation are common within OCD (also called SO-OCD), but SO-OCD is consistently misdiagnosed by physicians and psychologists. To address this issue, we describe the development of a self-report measure for assessing SO-OCD to help distinguish OCD from distress caused by a sexual orientation identity crisis. The current paper details two studies that established the psychometric properties and clinical utility of this measure. In Study 1, the factor structure, validity, and reliability were examined for the measure’s 12 items in a sample of 1,673 university students. The results revealed a two-factor solution for the measure (Factor 1: Transformation Fears; Factor 2: Somatic Checking) and preliminary evidence of validity and reliability. In Study 2, the measure was tested with LGBTQ and heterosexual community samples and clinical samples of individuals with SO-OCD and other types of OCD. The two-factor solution and evidence of validity and reliability were supported in these samples. Cut-off points were established to distinguish between community members and SO-OCD sufferers, as well as between those experiencing SO-OCD and other types of OCD. Limitations and future directions are discussed.     

Rorschach score validation as a model for 21st-century personality assessment

Recent conceptual and methodological innovations have led to new strategies for documenting the construct validity of test scores, including performance-based test scores. These strategies have the potential to generate more definitive evidence regarding the validity of scores derived from the Rorschach Inkblot Method (RIM) and help resolve some long-standing controversies regarding the clinical utility of the Rorschach. After discussing the unique challenges in studying the Rorschach and why research in this area is important given current trends in scientific and applied psychology, I offer 3 overarching principles to maximize the construct validity of RIM scores, arguing that (a) the method that provides RIM validation measures plays a key role in generating outcome predictions; (b) RIM variables should be linked with findings from neighboring subfields; and (c) rigorous RIM score validation includes both process-focused and outcome-focused assessments. I describe a 4-step strategy for optimal RIM score derivation (formulating hypotheses, delineating process links, generating outcome predictions, and establishing limiting conditions); and a 4-component template for RIM score validation (establishing basic psychometrics, documenting outcome-focused validity, assessing process-focused validity, and integrating outcome- and process-focused validity data). The proposed framework not only has the potential to enhance the validity and utility of the RIM, but might ultimately enable the RIM to become a model of test score validation for 21st-century personality assessment.     

哪个测验Q矩阵更合理:基于DINA模型测验Q矩阵合理性侦查指标及其比较与应用

本研究对多个测验Q矩阵的相对合理性的比较与选用开展研究,采用Monte Carlo模拟与实证研究相结合的范式,探讨R_square、HCI、-2LL、AIC、BIC、residual、ABS_residual及本研究新开发的BIC2等八项指标在测验Q矩阵合理性侦查效果及其比较。研究发现:八项指标中,除BIC和BIC2两项指标的对测验Q矩阵相对合理性的平均正确识别率在95%以上,其余指标的平均正确识别率不足90%,整体而言,考虑样本容量及参数个数双重加权的BIC和BIC2两项指标的表现总体上优于其它几项指标;各项指标在不同Q矩阵错误类型下其正确识别率也不尽相同。     

Optimism, self-mastery, and symptoms of depression in women professionals

The construct validity and predictive utility of dispositional optimism were examined in a sample of 192 women professionals. By using covariance structure modeling with latent variables, opotimism (Scheier & Carver, 1985) and self-mastery (Pearlin & Schooler, 1978) were found to be empirically distinct, although substantially correlated, constructs. Furthermore, although optimism and self-mastery were significant and negatively correlated with symptoms of depression, only self-mastery was independently associated with symptom levels. In addition, no evidence was found that optimism and self-mastery interact to influence depressive symptoms. These results suggest that the apparent predictive power of optimism may derive from its substantial overlap with self-mastery. Implications for the assessment and interpretation of optimism and self-mastery are discussed.     

Conscientiousness and self-motivation as mutually compensatory predictors of university-level GPA

To help account for variability across studies in the predictive utility of conscientiousness, we proposed that conscientiousness and self-motivation mutually compensate for each other in predicting university-level academic performance. Consistent with this expectation, we found evidence of such mutual moderation in a sample of 377 college undergraduates. First, we found that conscientiousness and self-motivation compensated for each other in predicting university GPA: Students who were either high in conscientiousness or high in self-motivation had better academic performance (GPA) than those who were low in both conscientiousness and self-motivation. Second, these findings were still evident after we controlled for the students’ previous academic performance (high school rank) and academic ability (SAT/ACT). The study of mutually compensatory predictors not only offers the potential of developing better predictive models; it also helps to account for why some “main effect” predictors of university GPA are variable across studies in their degree of predictive utility.     

Mutual Help Groups for Mental Health Problems: A Review of Effectiveness Studies

This paper reviews empirical studies on whether participating in mutual help groups for people with mental health problems leads to improved psychological and social functioning. To be included, studies had to satisfy four sets of criteria, covering: (1) characteristics of the group, (2) target problems, (3) outcome measures, and (4) research design. The 12 studies meeting these criteria provide limited but promising evidence that mutual help groups benefit people with three types of problems: chronic mental illness, depression/anxiety, and bereavement. Seven studies reported positive changes for those attending support groups. The strongest findings come from two randomized trials showing that the outcomes of mutual help groups were equivalent to those of substantially more costly professional interventions. Five of the 12 studies found no differences in mental health outcomes between mutual help group members and non-members; no studies showed evidence of negative effects. There was no indication that mutual help groups were differentially effective for certain types of problems. The studies varied in terms of design quality and reporting of results. More high-quality outcome research is needed to evaluate the effectiveness of mutual help groups across the spectrum of mental health problems.     

Conscientiousness and self-motivation as mutually compensatory predictors of university-level GPA

To help account for variability across studies in the predictive utility of conscientiousness, we proposed that conscientiousness and self-motivation mutually compensate for each other in predicting university-level academic performance. Consistent with this expectation, we found evidence of such mutual moderation in a sample of 377 college undergraduates. First, we found that conscientiousness and self-motivation compensated for each other in predicting university GPA: Students who were either high in conscientiousness or high in self-motivation had better academic performance (GPA) than those who were low in both conscientiousness and self-motivation. Second, these findings were still evident after we controlled for the students’ previous academic performance (high school rank) and academic ability (SAT/ACT). The study of mutually compensatory predictors not only offers the potential of developing better predictive models; it also helps to account for why some “main effect” predictors of university GPA are variable across studies in their degree of predictive utility.