MCMI-III Diagnostic Validity: Bad Test or Bad Validity Study

Operating characteristics describe the validity of tests that attempt to dichotomously predict a diagnosis. These statistics are not fully published in the Millon Clinical Multiaxial Inventory-III Manual (MCMI-III manual Millon, 1994) When calculated from available statistics, the positive predictive powers of the MCMI-III scales are poor both in absolute terms and relative to the MCMI-II (Millon, 1987). There were a number of problems, however, with the initial MCMI-III validity study both inherently and in execution. Although it is doubtful that the MCMI-III is weaker than the MCMI-II, a new validity is needed.     

Applications of Job Component/Synthetic Validity to Construct Validity

The origin and history of job component validity (JCV) and its relationship to construct validity are presented, followed by a summarization of how the methodology has been accomplished in research using the Position Analysis Questionnaire (PAQ). Data are presented that document the capability of both PAQ-based attribute profiles and predictions of aptitude data from PAQ job dimensions to establish job requirements. Finally, two practical ap- plications of JCV are described that illustrate the unique capability of the methodology to develop selection systems to screen individuals for jobs un- dergoing design or occurring in difficult-to-study environments.     

Research Validity Scales for the NEO-PI-R: Additional Evidence for Reliability and Validity

We examined the reliability and validity of the research validity scales (Schinka, Kinder, & Kremer, 1997) for the NEO-Personality Inventory-Revised (NEO-PI-R) in a clinical sample. The Negative Presentation Management (NPM) and Positive Presentation Management (PPM) scales were found to have satisfactory internal consistency reliability. Support for the validity of these scales was provided by the pattern of convergent and discriminant correlations with respective Personality Assessment Inventory (PAI) validity scales. Finally, PAI profiles of individuals with invalid NPM scores were found to differ significantly from those with valid NPM scores. Comparisons of the invalid profiles with profiles from other clinical samples provided additional support for the use of the NPM scale as a measure of negative impression management.     

Validity and Necessity

In this paper I argue against the commonly received view that Kripke’s formal Possible World Semantics (PWS) reflects the adoption of a metaphysical interpretation of the modal operators. I consider in detail Kripke’s three main innovations vis-à-vis Carnap’s PWS: a new view of the worlds, variable domains of quantification, and the adoption of a notion of universal validity. I argue that all these changes are driven by the natural technical development of the model theory and its related notion of validity: they are dictated by merely formal considerations, not interpretive concerns. I conclude that Kripke’s model theoretic semantics does not induce a metaphysical reading of necessity, and is formally adequate independently of the specific interpretation of the modal operators.     

Validity of self-reported fear

This study investigated the relationship between a self-report fear measure and several measures of fear made in the fear-arousing situation. Subjects at three levels of self-reported fear of spiders (mild, moderate and intense) were found to differ on most of the situational measures from subjects who had professed no spider fear. Subjects who initially reported different levels of spider fear did not differ among themselves on the situational measures. It was concluded that self-report paper and pencil measures of fear can be valid, though gross, indicators of fear behavior in an actual situation.     

When Validity Testing Lacks Validity: Comment on Levine et al.

The death of self‐construal research has been greatly exaggerated. Levine et al. (2003) highlight the alleged methodological limitations of the current body of the self‐construal model of culture. Their allegations, whether true or not, require a fair investigation. Careful examination of Levine et al. reveals pervasive logical flaws, methodological errors, and interpretation biases, which stem from ethnic stereotyping and other erroneous assumptions at various junctures. These fundamental errors, in turn, render their conclusions untenable. In testing their 1st set of hypotheses, which they proclaim to be “central to the validity” of self‐construal scales, Levine et al. bifurcate the participants into “Westerners” versus “Asians,” and then impose their a priori stereotype of how Westerners ought to be versus how Asians ought to be, which is scientifically unacceptable. This is an unscientific validation criterion (comparable to crude ethnic stereotypes such as “Asians are smarter”) and should not be used to judge the scales' validity. Next, in their discussion of priming, Levine et al. assert that the construct validity of the interdependent self‐construal requires that it must be susceptible to priming. This is an elementary logical fallacy. It simply does not follow from the definition of the interdependent self‐construal and therefore cannot be used as a criterion which reflects on construct validity. Their priming experiments are also deeply flawed by the unrepresentative sample of participants. Further, in their factor analyses, Levine et al. set up the “straw man” of a “universal a priori 2‐factor solution” although no self‐construal scholar or theory insists that there are precisely 2 universally applicable self‐construals or that the current self‐construal scales are perfect. The results of statistical analyses, such as those in Levine et al., crucially depend on the selection of presuppositions. Levine et al.'s presuppositions are untenable, yet they are passed off as self‐evident criteria for validity testing. Their article tries to create an illusion of finality, but is pervasively and fundamentally flawed.     

Construct Validity of Belief Systems Subscales

Abstract

The purpose of this study was to evaluate the construct validity of 11 subscales from the Belief Systems Test, a new Likert-type measure of conceptual system, as defined by Harvey, Hunt, and Schroder's (1961) model of cognitive development. Using responses from 136 undergraduate college students, we examined (a) intercorrelations among scores on the subscales, and (b) correlations between scores on the subscales and several measures of stress, an affective trait theoretically and empirically related to several conceptual systems. The scales associated with System 2 (characterized by rebelliousness and a distrust of people) and those associated with System 3 (defined by a concern for social relationships) emerged as the most cohesive sets of beliefs. System 2 subscales were also positively related to each of the five measures of stress administered.     

效度概化的理论研究述评

效度概化（validity generalization：VG）是心理测量理论和元分析的结合体,也是元分析在人事测评领域的重要应用。它深刻地挑战了人事测评效度的情景特异论,在工业与组织心理学的发展历程当中具有里程碑意义。该文系统地介绍了效度概化理论的逻辑思路、主要模型、关键理论语与应用,以及研究趋势。     

Construct Validity of the Acquired Capability for Suicide Scale: Factor Structure,Convergent and Discriminant Validity

The Acquired Capability for Suicide Scale (ACSS) assesses one of three main constructs in the Interpersonal Theory of Suicide but evidence of its validity is limited. In two studies (Ns = 287 and 738) validity of the full 20-item ACSS and its shorter versions (ACSS-5, ACSS-8, ACSS-FAD) were examined in terms of factor structure and relation to indices of self-reported suicidal behaviour and self-harm. Confirmatory factor analysis (CFA) failed to show good fit for one-, two-, or three-factor models of the ACSS in its various versions. Exploratory factor analysis of the 20-item scale in the first study pointed instead to a five-factor structure and this was supported using CFA in the second study. In both studies all scale versions showed moderate negative correlations to fear of death and dying, indicating scale validity for the purpose of assessing fearlessness about death. In the second study, a model in which the five factors were indicators of a latent variable of Capability was found to predict a latent variable of Suicidality as indicated by suicidal behaviours, but the prediction was substantially enhanced by the addition of Item 20 to the model. This single item was also found in the first study to better predict suicidal and self-harming behaviour than the full ACSS or any of its short versions.     

Estimating Incremental Validity Under Missing Data

A common form of missing data is caused by selection on an observed variable (e.g., Z). If the selection variable was measured and is available, the data are regarded as missing at random (MAR). Selection biases correlation, reliability, and effect size estimates when these estimates are computed on listwise deleted (LD) data sets. On the other hand, maximum likelihood (ML) estimates are generally unbiased and outperform LD in most situations, at least when the data are MAR. The exception is when we estimate the partial correlation. In this situation, LD estimates are unbiased when the cause of missingness is partialled out. In other words, there is no advantage of ML estimates over LD estimates in this situation. We demonstrate that under a MAR condition, even ML estimates may become biased, depending on how partial correlations are computed. Finally, we conclude with recommendations about how future researchers might estimate partial correlations even when the cause of missingness is unknown and, perhaps, unknowable.     

Social Status: Construct and External Validity

Abstract

The present study examined the construct and external validity of social status based on data covering various aspects of the construct, and collected from adolescent samples in Sweden, Australia, and the United States. Using correlation and factor analytic techniques it was found that (a) the various social status variables were, in general, only moderately related within the different countries, (b) the relations among variables were not congruent between countries, (c) two social status factors were revealed in all countries, separating educational-occupational status from social-economic status, and (d) only the educational-occupational factor showed high congruence between all of the countries.     

Self‐Construal Scales Lack Validity

Self‐construal is thought to mediate and explain the effects of culture on a wide variety of outcome variables. A meta‐analysis of published cross‐cultural self‐construal research is reported in this article, and the results across studies suggests that the evidence for the predicted cultural differences is weak, inconsistent, or nonexistent. The results of 3 priming experiments (N = 121, N = 99, and N = 361) suggest that (a) priming does not account for the inconsistent results observed in the meta‐analysis, (b) that scores on a self‐construal scale appear to be measuring trait‐like constructs that are not sensitive to priming, and (c) that measures of self‐construals lack convergent validity. The results of several measurement studies (N = 121, 223, 230, 323, 214, 206, 126, 204, 148, 141, and 150) were inconsistent with the a priori two‐factor measurement model in every case. Self‐construal scales were found to be radically multidimensional and highly unstable within and across cultures. These results lead us to conclude that catastrophic validity problems exist in research involving the use of self‐construal scales in cross‐cultural research.     

Validity of the GDS-4 revisited

This article points out several flaws in an earlier article (Chau, Martin, Thompson, Chang, & Woo, 2006). We note that Chau, Martin, Thompson, Chang, and Woo (2006) had misquoted our work on a 4-item version of the geriatric depression scale (GDS), and the work of the research team, which developed the original 30-item and 15-item versions of the scale. Furthermore, their data analytic methods were flawed, and their conclusions were often not supported by the data they presented. On the basis of these observations, we found no evidence against the use of the 4-item version of the GDS.     

Validity and quantification in intuitionism

We distinguish three different readings of the intuitionistic notions of validity, soundness, and completeness with respect to the quantification occurring in the notion of validity, and we establish certain relations between the different readings. For each of the meta-logical notions considered we suggest that the most natural reading (which is not the same for all cases) is precisely the one which is required by the recent intuitionistic completeness theorems for IPC.This paper was partly written while the first author, H. C. M. de Swart, was a Visiting Fellow at Princeton University for the academic year 1976–1977. This opportunity was given to him by the Niels Stensen Stichting in Amsterdam, the department of philosophy of the Catholic University in Nijmegen and by Princeton University. He is happy to express his appreciation for that opportunity.     

效度概化：预测效度元分析的30年成果述评

效度概化是通过元分析技术对普遍化的预测效度的估计。元分析是研究效度概化问题的技术,是对具有“预测因子-效标”特征的相关数据进行定量综合的方法。效度概化促进了预测因子与效标之间关系的理论研究和应用研究,是应用心理学领域近30年（1977~2007）来最重要的进展之一。30年来的效度概化研究表明,认知能力测验、知识和技能测验、人格测验、结构化面试和评价中心技术等的预测效度具有鲁棒性、对应性和联合增值性     

The Institutionality Of Legal Validity

The most influential theory of law in current analytic legal philosophy is legal positivism, which generally understands law to be a kind of institution. The most influential theory of institutions in current analytic social philosophy is that of John Searle. One would hope that the two theories are compatible, and in many ways they certainly are. But one incompatibility that still needs ironing out involves the relation of the social rule that undergirds the validity of any legal system (H.L.A. Hart's rule of recognition) to Searle's notion of codification: the idea that institutions need official declarations of their constitutive rules in order to enjoy the full benefits of institutions. The incompatibility arises from the fact that, in order to do its institutional work, the basic validity rule must be codified in Searle's sense—yet, given the particular role it has in legal positivism, it may be impossible to codify in the Searlean sense. In this paper I develop the incompatibility in detail, consider and reject consigning the basic validity rule to Searle's “Background” capacities that support institutional facts, and conclude that the best route to eliminating it while doing a minimum of damage to the two theories is to make a slight emendation to Searle's theory of institutions.     

Situational Tests in Student Selection: An Examination of Predictive Validity,Adverse Impact,and Construct Validity

The Flemish Admission Exam ‘Medical and Dental Studies’ is comprised of four cognitive ability tests and four situational tests, namely two work samples (i.e., a lecture and a medical text) and two video‐based situational judgement tests (i.e., a physician–patient interaction and a medical expert discussion). On the basis of the Admission Exam scores of 941 candidates (359 men, 582 women) this study shows that situational tests significantly can predict better than cognitive ability tests, with lecture and text emerging as significant predictors. When situational tests are combined with cognitive ability tests, there are no mean gender differences. Situational tests also enable us to measure a broader range of constructs. For example, in this study, the personality factor Openness is related to better situational test performance. Overall, this study demonstrates that situational tests may be a useful complement to traditional student selection procedures.     

Validity of the decisional process inventory

Empirical research on career indecision has been criticized for lacking theoretical conceptualization. To address this problem we examined the validity of the decisional process inventory (DPI), a measure of career decision making derived from a model based in Gestalt psychotherapy theory. Undergraduate college students (91 women, 110 men) responded to the DPI and criterion measures of career indecision and vocational identity. Results supported hypothesized relations between and among DPI items and subscales, and the ability of the measure to differentiate individuals by expressed career choice status. Significant intercorrelations among DPI subscales and criterion measures supported the concurrent validity of the DPI. Confirmatory maximum likelihood factor analysis of the DPI items did not support the hypothesized structure of the DPI to measure three career decision-making process dimensions: readiness, action, and resistance. Although results supported the basic psychometric properties of the DPI as a brief measure of career decision making, additional research is needed to clarify the underlying factor structure of the instrument.     

Construct Validity of Parent Ratings of Inhibitory Control

Recent literature has emphasized the need to examine executive functions (EF) in children using multiple sources, including both parent rating and performance-based measures. Computerized Go/No-Go tests, including commercially available continuous performance tests (CPTs), represent one of the most commonly used methods of assessing inhibitory control—a variable central to the executive function construct. We examined the relationship between parent ratings of inhibitory control and CPT performance in two mixed clinical samples. Experiment 1 examined 109 children ages 6–18 using the Behavior Rating Inventory of Executive Function (BRIEF; Gioia, Isquith, Guy, & Kenworthy, 2000) and the Conners' CPT-II (Conners, 2000). In this sample, ratings on the BRIEF Inhibit scale (mean T-score?=?62.3) were significantly higher than the CPT-II commissions score (mean T-score?=?50.7; p < .0001); and the BRIEF and CPT-II scores were not highly correlated (r?=???.12). Experiment 2 examined a sample of 131 children ages 7–18 using the BRIEF and the Tests of Variables of Attention (TOVA; Greenberg, 1996). In this sample, parent ratings on the BRIEF Inhibit scale (mean T-score?=?56.8) were similar to TOVA commissions scores (mean T-score?=?58.6; p?=?.33), although still poorly correlated (r?=??.02). Factor analyses exploring covariance between BRIEF scales CPT-II variables (Experiment 1) and between BRIEF and TOVA (Experiment 2) yielded similar findings. In both experiments, all eight BRIEF scales loaded on a single factor, with no overlap with either the CPT-II or the TOVA. In mixed outpatient clinical samples, the BRIEF appears to measure different elements of inhibitory control than those assessed by computerized continuous performance tests.     

The Validity of the Bender-Gestalt Emotional Indicators

We evaluated the concurrent validity of the revised Bender-Gestalt emotional indicators (Koppitz, 1975) among three groups of children in the 7- to 10-year age range: adjustment disorder, behavior disorder, and normal control. The concurrent validity of the emotional indicators total score was demonstrated. Notably, with the developmental level controlled, both clinical groups produced significantly more emotional indicators than the control group, although the two clinical groups could not be differentiated. Nominal-data analyses of the 13 individual emotional indicators indicated that three specific indicators (confused order, large size, boxed figures) possessed validity as pathognomonic signs. We discuss issues concerning the use of the emotional indicators in clinical practice.