Conceptual Complexity and Construct Validity

Despite a century of methodological and conceptual advances in the technology of psychosocial measurement, poor correspondence between indicators and the constructs they are intended to represent remains a limiting factor to the accumulation of scientific knowledge. Longstanding conventions in measurement may contribute to the failure to develop optimal criteria. These conventions include the focus on complex over simple constructs and the use of multi-item measures of disparate content to represent those constructs. Several arguments suggest that such a measurement model compromises the potential for developing measures that accurately reflect psychosocial phenomena. The article concludes with some preliminary suggestions concerning an alternative model that may address this construct validity problem more effectively.     

Social Status: Construct and External Validity

Abstract

The present study examined the construct and external validity of social status based on data covering various aspects of the construct, and collected from adolescent samples in Sweden, Australia, and the United States. Using correlation and factor analytic techniques it was found that (a) the various social status variables were, in general, only moderately related within the different countries, (b) the relations among variables were not congruent between countries, (c) two social status factors were revealed in all countries, separating educational-occupational status from social-economic status, and (d) only the educational-occupational factor showed high congruence between all of the countries.     

Applications of Job Component/Synthetic Validity to Construct Validity

The origin and history of job component validity (JCV) and its relationship to construct validity are presented, followed by a summarization of how the methodology has been accomplished in research using the Position Analysis Questionnaire (PAQ). Data are presented that document the capability of both PAQ-based attribute profiles and predictions of aptitude data from PAQ job dimensions to establish job requirements. Finally, two practical ap- plications of JCV are described that illustrate the unique capability of the methodology to develop selection systems to screen individuals for jobs un- dergoing design or occurring in difficult-to-study environments.     

RBANS Validity Indices: a Systematic Review and Meta-Analysis

Neuropsychology practice organizations have highlighted the need for thorough evaluation of performance validity as part of the neuropsychological assessment process. Embedded validity indices are derived from existing measures and expand the scope of validity assessment. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) is a brief instrument that quickly allows a clinician to assess a variety of cognitive domains. The RBANS also contains multiple embedded validity indicators. The purpose of this study was to synthesize the utility of those indicators to assess performance validity. A systematic search was completed, resulting in 11 studies for synthesis and 10 for meta-analysis. Data were synthesized on four indices and three subtests across samples of civilians, service members, and veterans. Sufficient data for meta-analysis were only available for the Effort Index, and related analyses indicated optimal cutoff scores of ≥1 (AUC?=?.86) and?≥?3 (AUC?=?.85). However, outliers and heterogeneity were present indicating the importance of age and evaluation context. Overall, embedded validity indicators have shown adequate diagnostic accuracy across a variety of populations. Recommendations for interpreting these measures and future studies are provided.     

Construct Validity of the Perceived Criticism Measure

The construct validity of the Perceived Criticism Measure (PCM) was examined in 2 studies. In Study 1, 50 community couples participated in problem-solving interactions after which they rated interaction-specific perceived criticism and their criticism of their spouses. In addition, they provided ratings of perceived criticism for their relationship overall and completed measures of psychopathology and marital satisfaction. For both husbands and wives, convergent validity was demonstrated by moderate-to-large correlations between the PCM and spouses' own ratings of their criticism for both general and interaction-specific perceived criticism. In Study 2, 37 patients with obsessive-compulsive disorder and their spouses participated in problem-solving interactions and provided ratings of marital satisfaction and general perceived criticism. Five untrained coders rated the interactions according to their own definitions of the relatives' destructive criticism of the patient. Their aggregated ratings proved strongly related to patients' PCM scores. Higher PCM scores were related to lower marital satisfaction in both Studies 1 and 2. The results of these studies are supportive of the convergent validity of the Perceived Criticism Measure. Evidence of discriminant validity was mixed.     

Construct Validity of the Iowa Gambling Task

The Iowa Gambling Task (IGT) was created to assess real-world decision making in a laboratory setting and has been applied to various clinical populations (i.e., substance abuse, schizophrenia, pathological gamblers) outside those with orbitofrontal cortex damage, for whom it was originally developed. The current review provides a critical examination of lesion, functional neuroimaging, developmental, and clinical studies in order to examine the construct validity of the IGT. The preponderance of evidence provides support for the use of the IGT to detect decision making deficits in clinical populations, in the context of a more comprehensive evaluation. The review includes a discussion of three critical issues affecting the validity of the IGT, as it has recently become available as a clinical instrument: the lack of a concise definition as to what aspect of decision making the IGT measures, the lack of data regarding reliability of the IGT, and the influence of personality and state mood on IGT performance.     

The Interaction Anxiousness Scale: Construct and Criterion-Related Validity

This article presents data regarding the validity and reliability of the Interaction Anxiousness Scale (LAS; Leary 1983c), a self-report measure of dispositional social anxiety. The IAS demonstrates high test-retest and internal reliability. Correlations with measures relevant to social and general anxiety document its convergent and discriminant validity, and it correlates well with measures of anxiety and interpersonal concern in actual interactions.     

Construct Validity of Parent Ratings of Inhibitory Control

Recent literature has emphasized the need to examine executive functions (EF) in children using multiple sources, including both parent rating and performance-based measures. Computerized Go/No-Go tests, including commercially available continuous performance tests (CPTs), represent one of the most commonly used methods of assessing inhibitory control—a variable central to the executive function construct. We examined the relationship between parent ratings of inhibitory control and CPT performance in two mixed clinical samples. Experiment 1 examined 109 children ages 6–18 using the Behavior Rating Inventory of Executive Function (BRIEF; Gioia, Isquith, Guy, & Kenworthy, 2000) and the Conners' CPT-II (Conners, 2000). In this sample, ratings on the BRIEF Inhibit scale (mean T-score?=?62.3) were significantly higher than the CPT-II commissions score (mean T-score?=?50.7; p < .0001); and the BRIEF and CPT-II scores were not highly correlated (r?=???.12). Experiment 2 examined a sample of 131 children ages 7–18 using the BRIEF and the Tests of Variables of Attention (TOVA; Greenberg, 1996). In this sample, parent ratings on the BRIEF Inhibit scale (mean T-score?=?56.8) were similar to TOVA commissions scores (mean T-score?=?58.6; p?=?.33), although still poorly correlated (r?=??.02). Factor analyses exploring covariance between BRIEF scales CPT-II variables (Experiment 1) and between BRIEF and TOVA (Experiment 2) yielded similar findings. In both experiments, all eight BRIEF scales loaded on a single factor, with no overlap with either the CPT-II or the TOVA. In mixed outpatient clinical samples, the BRIEF appears to measure different elements of inhibitory control than those assessed by computerized continuous performance tests.     

The Construct Validity of Assessment Centre Performance Dimensions

Evidence regarding the construct validity of assessment centre performance dimensions is reviewed. The evidence strongly suggests that variance in ratings tends to reflect exercises more than individual performance dimensions, thus calling into question the construct validity and utility of these dimensions. A number of biases in the assessment centre process, as well as more general rating biases are noted that may be responsible for these pervasive exercise effects. Suggestions are made for enhancing the construct validity of performance dimensions.     

The Validity of the Employment Interview: A Meta-Analysis

The interview is the most widely used personnel selection method, but has revealed low reliability and validity compared with other selection methods (Mayfield 1964; Ulrich and Trumbo 1965; Schmitt 1976; Arvey 1979). Thirty-one studies on the validity of the interview were meta-analyzed. The result was an average validity coefficient of .27. The estimated true validity of the interview was calculated to be .38. These values indicated that the interview has moderate validity as a personnel selection device. Six characteristics of the interview were also examined in relation to the validity of the interview: structure of the interview; number of interviewers; length of the interview; gender of the applicant pool; blue-collar/white-collar jobs; and use of college students versus job applicants. The six study characteristics accounted for 30.9% of the variance in the validity of the interview. Structure of the interview appeared to be the only characteristic that moderated the validity of the interview. The relationship of this study to other meta-analyses of the employment interview is discussed.     

A Meta-Analysis of the Convergent Validity of Self-Control Measures

There is extraordinary diversity in how the construct of self-control is operationalized in research studies. We meta-analytically examined evidence of convergent validity among executive function, delay of gratification, and self- and informant-report questionnaire measures of self-control. Overall, measures demonstrated moderate convergence (r_random = .27 [95% CI = .24, .30]; r_fixed = .34 [.33, .35], k = 282 samples, N = 33,564 participants), although there was substantial heterogeneity in the observed correlations. Correlations within and across types of self-control measures were strongest for informant-report questionnaires and weakest for executive function tasks. Questionnaires assessing sensation seeking impulses could be distinguished from questionnaires assessing processes of impulse regulation. We conclude that self-control is a coherent but multidimensional construct best assessed using multiple methods.     

Factorial and Construct Validity of FACES IV Among Italian Adolescents

The aims of this study were to evaluate the reliability, validity and factorial structure of Family Adaptability and Cohesion Evaluation Scale, Version IV (FACES IV), investigate the relevance of sex and age on family functioning and study the relationship between family functioning and adolescents’ perception of parent and peer attachment. The sample consisted of 1,416 participants (672 women, 744 men, M_age = 17.38 years; age range = 13–22 years) divided into three age groups: 13–15, 16–18, and 19–22 years. Factor analysis confirmed the dimensions formulated to define the construct of family functioning: all the items of each dimension were loaded in the same factor with a correlation of at least 0.30. Groups differed with regard to the age on Cohesion, Enmeshed and Rigid scales: Younger adolescents (age 13–15 years) showed significantly higher scores than adolescents (age 16–18 and 19–22). Sex differences revealed that females had a higher mean score on Cohesion and Flexibility, whereas males had a higher mean score on Disengaged, Enmeshed and Chaotic. Correlations between FACES IV and Parent and Peer Attachment Scale demonstrated the convergent validity of the FACES IV. Data confirmed the validity of FACES IV’s Italian version and suggested that the Circumplex Model has a cross-cultural applicability and it could be a useful instrument in the Italian context.     

Clarifying Construct Validity: Definitions,Processes, and Models

Despite definitions in standard sources, personnel managers, psychologists, and psychometricians persistently encounter problems that are best referred to as the ambiguous nature of validity. The purpose of this article is to pro- vide an overview of construct validity and personnel testing, to demonstrate its practical utility, and to clarify with concrete examples certain theories and models, as well as to illustrate the meaning of the terminology used by com- mentators on the topic. A brief historical overview of testing and validation is presented; the progress of construct validity and its acceptance by various sec- tors of society is discussed parsimoniously in the section The Seven Wonders of Personnel Psychology. In the past, personnel psychologists have not done a very good job of understanding the constructs that underlie test perform- ance. Some new approaches can help to correct this. A process should be rou- tinely used on all tests in order to develop an understanding of the constructs that underlie performance on an employment test; only by knowing the cor- rect criterion and method of measuring it can we ascertain the intrinsic valid- ity of our measures.     

The Construct Validity of Acceptance: A Multitrait-Multimethod Investigation

Despite increasing clinical and empirical attention, the construct validity of acceptance has not been extensively investigated. The present study utilized a multitrait-multimethod design and a correlated trait-correlated method minus one [CT-C(M-1)] confirmatory factor analytic model to assess acceptance's convergent validity across methods and discriminant validity in comparison to cognitive reappraisal and perceived emotional control in a sample of 210 outpatients with anxiety and mood disorders. In addition, the study evaluated acceptance's concurrent validity by investigating the extent to which it was associated with variables of clinical interest over and above the two rival constructs. Results of confirmatory factor analyses supported acceptance's convergent and discriminant validity in comparison to the two neighboring constructs, and thereby provided partial support for its construct validity. However, contrary to prediction, acceptance was not significantly associated with concurrent validation measures. These results are discussed in the context of acceptance's potential therapeutic utility and functional relationships with associated constructs.     

An External Construct Validity Study of Rorschach Personality Variables

This study examined (a) hypothesized relationships between Rorschach variables and self-report test measures relating to nominally similar aspects of personality functioning and (b) interrelationships among Rorschach variables. Sixty-two undergraduates were administered the Rorschach, Barron Ego Strength Scale, Kaplan Self-Derogation Scale, Eagly Self-Esteem Scale, Multiple Affective Adjective Checklist (MAACL), Marlowe-Crowne Social Desirability Scale, and the Rotter Locus of Control Scale. Only a few of the predictions received confirmation: inanimate movement (m) correlated, as expected, with MAACL anxiety and hostility, the egocentricity index (3r + 2)/R (R = total responses) correlated significantly with self-esteem, and human movement with minus form level (M -) correlated (inversely) with ego strength. Among the unpredicted findings were some that appear inconsistent with standard Rorschach interpretation. Rorschach variables human movement (M), and experience actual (EA), generally interpreted as reflecting coping resources, related significantly with self-report measures of poor coping and of dysphoric affect. In general, the Rorschach appears better at identifying weaknesses in the ego rather than strengths.     

Response Style: A Basis for Rorschach Construct Validity

The Rorschach test situation was defined as an ambiguous power relationship in which the S could respond either by moving away from (Recognition), moving toward (Reaction) or moving against (Proaction) the E. Construct validity was approached by predicting both to a trait (trait validity) and to a different theoretical system (nomological validity). To evaluate trait validity it was predicted that yielders would have more Reactive responses than non-yielders and that non-yielders would have more Proactive responses than yielders. Nomological validity was investigated by predicting into Schachter's (1959) theory of affiliation. First born and only children should produce more Reactive scores than late born children and late born children should produce more Proactive responses than first and only children. The prediction that the non-yielders would produce more Proactive scores than yielders was not sustained. However, the hypotheses about yielding and Reaction, first and only children and Reaction, and late children and Proaction were sustained.     

Examining the Construct Validity for the Multiple-Content Testing Programs

The primary objective of this study was to examine the construct validity for the 2 multiple-content testing programs-the multiple-choice Comprehensive Tests of Basic Skills (CTBS/5) together with the performance-based Maryland School Performance Assessment Program (MSPAP)-by evaluating the true-score longitudinal associations among multiple-content scores in 1 school district. Each content area measure in the 2 testing programs was closely examined by the following criterion: The true-score correlation between 2 time-period measures of the same content area is higher than its longitudinal true-score correlations with other content areas. This criterion was achieved in 2 (Reading and Mathematics) of 5 CTBS/5 content subtests, as well as 1 (Language) of 6 MSPAP content subtests. The structural equation modeling has been conducted on a multitrait-multimethod correlation dataset, where the traits of Reading and Mathematics were assessed by MSPAP and the old version of CTBS/4. Although convergent validity existed in these 2 measures, there was little evidence to support discriminant validity in both measures.     

The Driving Appraisal Inventory: Psychometric Characteristics and Construct Validity1

Four studies examined the dimensionality, reliability and construct validity of the Driving Appraisal Inventory (DAI), a self-report measure of driving habits and skills. In Study 1, analysis of 127 items led to the formation of four unidimensional, internally reliable scales: Carelessness, Drunken Driving, Vehicle Safety, and Self-Evaluation. In Study 2, all were found to have acceptable test-retest reliability. In Study 3, convergent validity support was obtained for the Carelessness and Vehicle Safety scales. In Study 4, concurrent criterion-related validity support was obtained for the Carelessness and Drunken Driving scales. Scores on the Self-Evaluation scale appear to be more a function of self-confidence and/or ego-involvement in driving skills and less a function of actual driving skills. Men score higher than women do on the Carelessness, Drunken Driving, and Self-Evaluation scales, and in Study 4, criterion-related validity of the Self-Evaluation scale was stronger among women than among men.     

Nonverbal Memory Assessment with Designs: Construct Validity and Clinical Utility

The evaluation of memory is often a principal objective in psychological testing, especially for older adults. Differentiation between subtypes of memory functioning is critical in making differential diagnoses and in predicting everyday functioning. In this paper, 23 tests using memory for designs and developed for clinical or research applications are reviewed. Overall, reliabilities are low, normative samples varied in size, and findings regarding the construct validity of design memory testing are mixed. Tests with higher reliability, large numbers of designs, and recognition formats may be more specific to nonverbal memory. A series of studies that systematically establishes construct validity for design memory tests will enhance clinical utility.