Large-sample confidence intervals for validity and reliability coefficients

Large-sample confidence intervals (CI) for reliability, validity, and unattenuated validity are presented. The CI for unattenuated validity is based on the Bonferroni inequality, which relies on one CI for test-retest reliability and one for validity. Covered are four reliability-validity situations: (a) both estimates were from random samples; (b) reliability was from a random sample but validity was from a selected sample; (c) validity was from a random sample but reliability was from a selected sample; and (d) both estimates were from selected samples. All CIs were evaluated by using a simulation. CIs on reliability, validity, or unattenuated validity are accurate as long as selection ratio is at least 20% and selected sample size is 100 or larger. When selection ratio is less than 20%, estimators tend to underestimate their parameters.     

A survey of neuropsychologists’ use of validity tests with children and adolescents

A web-based survey of validity test use by North American neuropsychologists was conducted, with 282 participants meeting inclusion criteria. Respondents indicated that they use a median of one stand-alone performance validity test (PVT), one embedded PVT, and one symptom validity test (SVT) per pediatric assessment. The vast majority of respondents indicated they give at least one PVT (92%) and at least one SVT (88%) during each pediatric assessment. A meaningful difference in validity use (i.e., at least a medium effect size) was only found for those who engage in forensic work, with those clinicians giving more stand-alone PVTs than those who do not conduct forensic work. The most frequently used validity measures in pediatric assessments are presented, as are reasons participants reported for both using and not using validity tests. Limitations and qualitative comparisons to other surveys on validity test use with adults are discussed.     

Personality Types and Applicant Reactions in Real‐life Selection

The study aimed to determine if an applicant's personality type is associated with his/her reactions (fairness perceptions, face validity perceptions, and predictive validity perceptions) to the selection process. The participants (N = 258) were real‐life applicants for admission to a vocational school. A person‐centered approach was applied to find subgroups with similar personality profiles. Latent profile analysis found four personality types: Resilient (45%), Overcontrolled (13%), Undercontrolled (10%), and Bohemian (32%). The Resilient and Bohemian personality types had more favorable perceptions of test fairness than the Overcontrolled type. Personality type did not affect face validity or predictive validity perceptions. The results suggest that personality (type) could be included in applicant reactions models and in analyses investigating the antecedents of applicant reactions.     

Validity rates of the Personality Assessment Inventory and the Minnesota Multiphasic Personality Inventory-2 in a VA Medical Center Setting

In this study, we compared protocol validity rates between the MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) and the Personality Assessment Inventory (PAI; Morey, 1991) in a veteran population. Veterans (N=472) were administered both instruments as part of routine psychological evaluations. Profile validity was based on previously published criteria. When applying primary validity indicators, inpatients produced significantly fewer invalid PAI profiles (37%) than MMPI-2 profiles (63%). We found similar results among outpatients for which we considered 47% of MMPI-2 profiles invalid compared to only 21% of PAI profiles. When applying both primary and supplementary validity indicators, both inpatients and outpatients continued to produce fewer invalid PAI profiles than MMPI-2 profiles. We discuss factors that may be related to the differences in validity rates.     

不确定性和预期有效性对运动方向感知决策的影响

环境中的运动刺激往往多变, 更好的方向感知需要同时依赖当前信息和先验信息。然而, 当前刺激的不确定性与先验线索的预期有效性在运动方向感知决策中如何整合尚不清楚。研究采用随机点运动范式, 要求被试判断散点群的整体运动方向, 方向一致的散点为相干点, 其比例越低, 不确定性越高。实验1比较了不同相干点比例下的感知准确度。根据实验1的结果, 选出20%和60%的相干点比例分别对应实验2中散点群的高、低不确定性水平。实验2比较了高、低不确定性和高、低预期有效性(线索正确预测目标的概率)下的感知准确度。结果发现, 随着不确定性的降低和预期有效性的增加, 感知准确度显著提高; 预期效应(高、低有效性下感知准确度的差异)随不确定性的提高而增加。结果表明, 先验信息与感觉信息能在运动方向感知决策中整合。研究为双重控制机制(DMC)理论下主动性与反应性控制协同工作的观点提供了实证支持, 并为交通事故的解释与预防提供了理论依据。     

个人拟合指数对人格测验的影响及意义

个人拟合指标是考察心理测验中偏差得分模式的新方法。研究中考察了G、C、MCI、U3、U、W、ECI6、L等8个拟合指标对艾森克人格问卷信效度的影响,以及各指标与正反向题回答不一致项目数的相关。结果表明,删除不同比例拟合程度不好的个体后,测验的信效度明显提高。同时PFS可鉴别人格测验中的默认反应偏差。各指标中l对测验信效度的改善效果最为理想。     

Predicting job performance using FFM and non‐FFM personality measures

This study compares the criterion validity of the Big Five personality dimensions when assessed using Five‐Factor Model (FFM)‐based inventories and non‐FFM‐based inventories. A large database consisting of American as well as European validity studies was meta‐analysed. The results showed that for conscientiousness and emotional stability, the FFM‐based inventories had greater criterion validity than the non FFM‐based inventories. Conscientiousness showed an operational validity of .28 (N = 19,460, 90% CV = .07) for FFM‐based inventories and .18 (N =5,874, 90% CV = ‐.04) for non‐FFM inventories. Emotional stability showed an operational validity of .16 (N = 10,786, 90% CV = .04) versus .05 (N = 4,541, 90% CV = ‐.05) for FFM and non‐FFM‐based inventories, respectively. No relevant differences emerged for extraversion, openness, and agreeableness. From a practical point of view, these findings suggest that practitioners should use inventories based on the FFM in order to make personnel selection decisions.     

The Validity of the Employment Interview: A Meta-Analysis

The interview is the most widely used personnel selection method, but has revealed low reliability and validity compared with other selection methods (Mayfield 1964; Ulrich and Trumbo 1965; Schmitt 1976; Arvey 1979). Thirty-one studies on the validity of the interview were meta-analyzed. The result was an average validity coefficient of .27. The estimated true validity of the interview was calculated to be .38. These values indicated that the interview has moderate validity as a personnel selection device. Six characteristics of the interview were also examined in relation to the validity of the interview: structure of the interview; number of interviewers; length of the interview; gender of the applicant pool; blue-collar/white-collar jobs; and use of college students versus job applicants. The six study characteristics accounted for 30.9% of the variance in the validity of the interview. Structure of the interview appeared to be the only characteristic that moderated the validity of the interview. The relationship of this study to other meta-analyses of the employment interview is discussed.     

Face validity of test and acceptance of generalized personality interpretations.

After students in different groups took one of four "personality tests" that varied in face validity, they received a generalized personality interpretation. Even though the test lowest in face validity merely required the subject to circle digits, judged accuracy of the interpretation (a) was high for all tests (76-87% of the ratings in the various test groups were good or excellent) and (b) was independent of type of test upon which the interpretation was purportedly derived. From 46-60% of the subjects in the various groups "definitely liked" or "liked" the interpretation; likability was independent of face validity of the test. Judged accuracy and likability of the interpretation correlated positively and significantly. The willingness of individuals to accept generalized statements about themselves is a researchable problem area suggested by the present research.     

The Structured Assessment of Violence Risk in Youth as a predictor of recidivism in a United Kingdom cohort of adolescent offenders with conduct disorder

This prospective study examined the predictive validity of the Structured Assessment of Violence Risk in Youth (SAVRY) and the Psychopathy Checklist: Youth Version (PCL: YV) in 99 male adolescents who were assessed in custody and followed up 12 months post release. Outcome data on recidivism were based on official Home Office records. The base rates for violent and general recidivism, respectively, were 38.4% and 70.7%. The predictive validity of the SAVRY Risk Total and the SAVRY Risk Rating was moderate for both violent and general recidivism, but both showed incremental validity in predicting outcomes compared with the PCL: YV. Data are discussed in relation to the limited published international literature.     

MMPI-2 typology of pregnant drug-dependent women in treatment

The present study examined the validity of Minnesota Multiphasic Personality Inventory-2 (MMPI-2) typology for pregnant drug-dependent women. A 3-cluster solution based on 7 MMPI-2 clinical scales emerged as the best model and was replicated across split-half samples and different primary substance-use diagnoses and treatment modalities. The 3 subtypes identified included Type I (n = 40, 24%) with no clinical elevation, Type II (n = 72, 42%) with elevated psychopathic deviate scale, and Type III (n = 58, 34%) with elevations on all 7 scales. Analyses with interview and self-report measures showed good concurrent validity. Type II had higher retention than Type I and Type III across methadone and medication-free treatments, showing some predictive validity. An a priori method for classifying new cases on the basis of the proposed typology was developed and validated. Study findings support MMPI-2's use with pregnant drug-dependent women for assessment and possibly treatment planning.     

The role of psychopathic personality disorder in violence risk assessments using the HCR-20

Antisocial and psychopathic traits are essential to evaluate when assessing risk for violence using the HCR-20. The role of the PCL-R on the HCR-20 was investigated using a series of meta-analytic tests. Across 34 samples in which both tools were rated, AUCs for violence were similar (～.69), and exclusion of the psychopathy item (H7) did not reduce the HCR-20's accuracy. Quantitative synthesis of results from multivariate analyses conducted in 7 raw datasets that used both tools demonstrated that the average probability of observing violence for every point increase on the HCR-20 (without H7), while controlling for the PCL-R, was 23%, whereas for the PCL-R it was -1%. The HCR-20 (without H7) added incremental validity to the PCL-R, whereas the converse was not true, and only the HCR-20 (without H7) possessed unique predictive validity. Results suggest the HCR-20's predictive validity was not negatively impacted by excluding the PCL-R. Areas for future study are discussed, including research on various ways to assess and incorporate into risk assessment personality traits related to violence.     

The depressive personality disorder inventory and current depressive symptoms: implications for the assessment of depressive personality

The Depressive Personality Disorder Inventory (DPDI; Huprich, Margrett, Barthelemy, & Fine, 1996; see Appendix) was created to assess Depressive Personality Disorder in clinical and nonclinical samples. Since its creation, the DPDI has been used in multiple studies, and the psychometric properties of the measure have generally supported its reliability, convergent validity, and construct validity; however, evidence for the measure's discriminant validity has been mixed. Specifically, the DPDI tends to correlate highly with measures of current depressive symptoms, which limits its efficacy in differentiating current depressive symptoms from a depressive personality structure. A principal components analysis of 362 individuals who completed both the DPDI and Beck Depression Inventory (BDI-II; Beck, Steer, & Brown, 1996) found that 49% of the variance was accounted for in two components. Seven items from the DPDI loaded more strongly on the first component composed of many BDI-II items. These items were removed in order to create a measure believed to assess DPD without the confounding influence of current depressive symptomology. Principal components analysis of the revised measure yielded three components, accounting for 46% of the variance. The revised DPDI was used to calculate convergent, discriminant, and construct validity coefficients from measures used in former studies. Virtually no improvement in the validity coefficients was observed. It is concluded that assessing DPD via self-report is limited in its utility.     

A New Measure of Anti-Arab Prejudice: Reliability and Validity Evidence

Two studies were conducted to provide reliability and validity support for a new anti-Arab prejudice scale. The scale was designed to fit to the European context and showed very satisfactory reliability. Moreover, both studies provided convergent validity support. Anti-Arab prejudice was correlated with authoritarianism, anti-Semitism, and conservatism. The correlation between the new scale and an adapted version of McConahay's (1986 ) Modern Racism scale was very strong. Furthermore, the second study provided predictive validity support. Scores in the new scale explained 20% of the variance in an ulterior actual behavior (to return a signed form supporting an association aimed to defend European values and culture against Islamization).     

VALIDITY GENERALIZATION: A REVIEW AND CRITIQUE OF THE CORRELATION MODEL

This paper presents a review and critique of the validity generalization research based on the correlation model. Emphasis was placed on integrating the applied validity generalization studies as well as the computer simulation studies which have tested the accuracy of the various validity generalization procedures in estimating the mean and variance of true validity coefficients. In general, this review indicated that the validity generalization procedures are fairly accurate in estimating the mean and variance of true validities, that the preponderance (e.g., 90%) of corrected validity coefficients for a single predictor-criterion relationship, based on a series of cumulative studies, when placed in distributional form are positive, and that a substantial proportion of observed validity coefficient variation can be attributed to statistical artifacts with sampling error accounting for the majority of the artifactual variance. Moreover, potential problems with validity generalization/meta analysis based on criterion-related validity coefficients were discussed. It was concluded that validity general-ization/meta-analysis is an important development in summarizing cumulative research, however, caution should be exercised in utilizing and interpreting the findings of such analyses. Directions for future research aimed at establishing trait-performance relationships were discussed.     

Big Five Personality Dimensions and Job Performance in Army and Civil Occupations: A European Perspective

This article reports three meta-analyses on the relation between the Big Five personality dimensions and job performance using exclusively European samples. Also, the incremental validity of Conscientiousness and Emotional Stability over General Mental Ability is reported. The results show that Conscientiousness (p = .23) and Emotional Stability (p = .23) generalized validity across criteria and civil and military occupations. The results also showed that Conscientiousness and Emotional Stability had incremental validity and added to total validity a percentage around 10% to 11 %. Implications for the international generalizability of validity and the practice of personnel selection are commented on.     

MCMI-II report style and bias: profile and validity scales analyses

This study examines the behavior of the Millon Clinical Multiaxial Inventory-II (MCMI-II) in the face of various response styles and biases. The profiles and validity configurations of eight different test-taking styles were analyzed. Four hundred MCMI-II inventories (50 for each of the following categories) were administered or generated to produce the following: (a) normal endorsement by subjects, (b) fake good for administrative reasons, (c) fake good for clinical reasons, (d) fake bad administratively, (e) fake bad clinically, (f) 50% true/50% false computer generated, (g) 95% true computer generated, and (h) 95% false computer generated. Good statistical and clinically relevant separation of the profiles was found for normal, fake good, fake bad, and the randomly generated profiles with 44% of the variance predicted. The percentage of profiles identified by validity scales, however, was modest.     

MCMI-II Report Style and Bias: Profile and Validity Scales Analyses

This study examines the behavior of the Millon Clinical Multiaxial Inventory-II (MCMI-II) in the face of various response styles and biases. The profiles and validity configurations of eight different test-taking styles were analyzed. Four hundred MCMI-II inventories (50 for each of the following categories) were administered or generated to produce the following: (a) normal endorsement by subjects, (b) fake good for administrative reasons, (c) fake good for clinical reasons, (d) fake bad administratively, (e) fake bad clinically, (f) 50% true/50% false computer generated, (g) 95% true computer generated, and (h) 95% false computer generated. Good statistical and clinically relevant separation of the profiles was found for normal, fake good, fake bad, and the randomly generated profiles with 44% of the variance predicted. The percentage of profiles identified by validity scales, however, was modest.     

The use of the peabody picture vocabulary test with emotionally disturbed adolescents

The validity and reliability for the Peabody Picture Vocabulary Test (PPVT) was estimated with 52 nonpsychotic emotionally disturbed adolescents. Approximately 6 months after an initial administration of the PPVT and the Peabody Individual Achievement Test (PIAT), all children were retested. Significant increases in PPVT IQ and PIAT reading comprehension scores were found upon readministration. The internal consistency estimate for the initial PPVT administration showed considerable consistency in responding. A modest temporal stability estimate indicated a fair amount of fluctuation in scores over time. Concurrent validity coefficients for the PPVT indicated moderate to dependable relationships with PIAT subtests. Predictive validity estimates for the PPVT suggested moderate relationships with future achievement, accounting for some 26% of the variability in total achievement.     

Concurrent evidence of physical educators' attitude toward teaching individuals with disabilities

This study assesses the criterion-related relevance or concurrent validity of Physical Educators' Attitude Toward Individuals with Disabilities III survey, developed in 1993 by Rizzo. The criterion measure was the Learning Handicapped Inventory, developed in 1982 by Jansma and Shultz, because it, like the former, has validity and reliability in assessing attitudes of physical educators toward teaching students with disabilities. Participants were 179 physical education majors attending a major midwestern university, 94 (53%) women and 85 (47%) men whose average age was 20.7 yr. (SD = 2.5 yr.). Both surveys were administered in one sitting, and the correlation of .88 indicates strong concurrent validity.