The selective efficiency of a test battery

In industrial acceptance sampling one frequently makes use of operating characteristic curves to describe the discriminating power of a particular sampling plan. Similarly, it is possible to demonstrate the selective efficiency of a test battery in terms of (a) the Applicant's Operating Characteristic (A.O.C.); (b) the Selector's Operating Characteristic (S.O.C.). The A.O.C. determines the chance of selection by means of a test for any given level of true ability. The S.O.C. connects functionally probability of success on the criterion with the predictor scores of a battery. For the case of a normal bivariate distribution the exact mathematical expressions of the OC curves are derived in terms of the correlation coefficient, the cut-off points and, and the predictor and criterion scoresX andY (in standard measures). The Efficiency IndexH is defined as the percentage of successful subjects gained by the use of a test battery, taking chance selection as a yardstick for comparison. Its optimum, for fixed and , is derived. The distribution law of the criterion scores of selectees is deduced and its first four moments are shown to depart little from normality for cases usually encountered in practice. A Quality-Gain diagram graphically illustrates the improvements secured. Another simple device, the Cost-Utility diagram, explains to management the full implications of selecting personnel by means of a test battery. Neither of the diagrams requires an understanding of the correlation coefficient. The confidence belt of the OC curves, the standard error of the mean criterion score of selectees and the standard error of the predicted number of successful applicants are determined. Finally, the full theory is applied in detail to a real test battery.This paper is the result of an investigation undertaken by the South African National Institute for Personnel Research, Johannesburg, and was completed at the Educational Testing Service, Princeton. The author wishes to thank these organizations for permission to publish this paper.     

The relation between the difficulty and the differential validity of a test

Using scores of 1200 students on a long test as a criterion, each of five subtests of different difficulty has maximum correlation with the criterion when the criterion is dichotomized at a value appropriate to the difficulty of the subtest. A 50-item test element is scored on an all-or-none basis with different standards for passing, and the percentage of passes for successive points on the criterion variable is computed. The Constant Method is applied to this relationship. The limen thus computed is a measure of difficulty, the dispersion is a measure of average (or total) validity, and the slope of the curve is a measure of differential validity. The difficulty of a test element is thus directly related to the maximum differential validity.     

The multiple-stimulus-without-replacement preference assessment tool and its predictive validity

This study demonstrates the use of two web-based programs, one to identify video preferences and the other to assess their reinforcing effects. We used the Multiple-Stimulus-Without-Replacement Preference Assessment Tool (MSWO PAT) to identify the video preference hierarchies of seven participants, ages 4–11 years old. We then used a customized reinforcer assessment program that arranged a concurrent-chains preparation with programmed conjugate schedules of reinforcement. Button presses emitted by participants modulated the quality (volume and opacity) of selected videos on a moment-to-moment basis, allowing us to identify the reinforcing effects of the videos in little time. The results showed that the preference assessment had predictive value for five of seven participants. We discuss the MSWO PAT, parameters that may affect the identification of preferences and the use of conjugate schedules to identify reinforcers.     

Evaluating the predictive validity of a single stimulus engagement preference assessment.

Most preference assessments require individuals to choose among two or more stimuli. More recently, the duration of engagement with items has been used as an index of preference. In the current study, the predictive validity of a single stimulus engagement (SSE) preference assessment was evaluated with 4 individuals. Stimuli were presented singly for a brief period while engagement with that stimulus was recorded. Although SSE preference rankings closely matched paired stimulus preference assessment rankings for only 2 of the 4 participants, relative preference rankings based on duration of engagement predicted relative reinforcer effectiveness for all participants in a subsequent concurrent‐schedule reinforcer assessment. The SSE procedure took less time to administer than the paired stimulus procedure but produced less stable preference rankings across administrations. The SSE procedure may be appropriate for individuals who have difficulty selecting one stimulus from among two or more stimuli, and it may be well suited for evaluating activities that are difficult to present in a paired stimulus format.     

The relationship between search efficiency and crowding

The role of target salience in crowding has remained controversial largely because salience usually escapes objective measurement. Here we address this problem using search efficiency as a measure of target salience. In separate experiments observers determined whether parafoveal arrays of vertical Gabor patterns contained targets having a unique colour, a unique direction of motion, and a unique temporal frequency. We analysed search efficiency in the conventional manner using reaction-time gradients (in seconds per item). We also considered accuracy gradients (in percent-correct per item). Crowding is typically quantified by comparing the acuity for a target within an array to the acuity for a target presented alone. We measured orientation acuity for determining whether a slightly tilted target was clockwise or anticlockwise of vertical. Targets with a unique colour or direction of motion were found to pop out, ie (with one exception) reaction-time and accuracy gradients were insignificantly different from zero. Acuity for these targets was significantly greater than acuity for targets whose neighbours had the same colour and direction of motion. Manipulation of temporal frequency produced a wide range of search efficiencies. For three of four observers we found a linear relationship between acuity and the accuracy gradient, shallow gradients being associated with high acuity. In general, we find that crowding is weakened when observers can find a parafoveally presented target quickly and accurately.     

The Kramer-Froehle controversy: a contribution to construct validity of the FIRO-B questionnaire

Replicated the conditions established by Kramer in his attempt to contribute to the construct validation of the FIRO-B. Froehle's apparent later replication produced significantly different results from Kramer's original study. In replicating Kramer's design this researcher wished to establish whether the earlier failure to replicate was due to a difference in design or to an actual lack of construct validity of the FIRO-B. Kramer's findings were supported and an alternative explanation for the difference in Kramer's and Froehle's findings is discussed.     

The validity of a persistence test

In order to raise the predictive efficiency of its college entrance test battery, the Educational Testing Service is working on the development of non-academic measures to supplement the standard aptitude and achievement examinations. A test of difficult number series problems was set up to measure persistence by tempting the students to give up early; the students were informed that some of the problems had no solution, and that full credit would be received by so marking them. This test was tried out and found to have some correlation with grades, while having no correlation with the other tests. Adding this test to the battery showed an appreciable rise in the battery's multiple correlation with grades.     

An ontological test of the IAT: Self-activation can increase predictive validity

Extensive research has been conducted demonstrating the predictive validity and reliability of the Implicit Association Test (IAT) for a broad array of behaviors and contexts. However, less work has been done examining its underlying construct validity. This contribution focuses on examining whether a core theoretical foundation of the IAT paradigm is valid, specifically, whether the IAT effect draws on the social knowledge structure (SKS). We present four studies within different domains that show that the IAT does indeed appear to draw on the SKS. The data show that activation of the self before the categorization task enhances the predictive validity of the IAT, as one would expect if the IAT reflects the SKS. We discuss theoretical reasons for these findings, with emphasis also on underlying statistical/psychometric issues.     

The contribution of personality characteristics to the relationship between social support and perceived physical health

Although a great deal of research has been conducted on the relationship between social support and physical health, the contribution of personality characteristics to this relationship has rarely been assessed. Structural equation modeling was employed to derive and test a model of the direct and indirect relationships between personality characteristics, social network size, the perceived availability of support, socially supportive behaviors, and perceived physical health with a sample of Pennsylvania adults. Significant paths indicate that individuals who perceive themselves as affiliative and as help seekers and help givers report larger social networks, receive more socially supportive behaviors, and perceive that more support is available to them. The perception that support is available shared a direct relationship with perceived physical health. The utility of including multiple measures of social support and personality characteristics related to receiving support in investigations of the relationship between social support and physical health is discussed.     

Construct validity of the relationship profile test: a self-report measure of dependency-detachment

This study evaluated the construct validity of the Relationship Profile Test (RPT; Bornstein & Languirand, 2003), a 30-item self-report measure of dependency-detachment that yields three subscale scores: (a) destructive overdependence, (b) dysfunctional detachment, and (c) healthy dependency. Scores on the RPT subscales generally showed the expected patterns of intercorrelations and gender differences, and comparison of RPT scores with scores on other tests supported the convergent and discriminant validity of each RPT subscale. Results of internal and retest reliability analyses were generally supportive as well, and suggested that the three RPT subscales assess aspects of the traits they purport to measure     

The predictive validity of subtle and obvious empirically derived psychological test items under faking conditions

The relative contributions of subtle and obvious item endorsements to the prediction of a relevant criterion were assessed under faking and control ("honest") conditions. The MMPI and a nonconformity questionnaire were first administered to 100 male college students. Items on the Pd scale and 101 additional MMPI items that correlated significantly with the nonconformity questionnaire were then rated by 38 other male college students for apparent relationship to psychopathology. From these ratings, a scale (designated PdX) was constructed, which consisted of 21 subtle and 21 obvious items. After a third group of 98 male college students completed the nonconformity questionnaire, they were asked to respond to the items of the Pd and PdX subscales under control, fake-good, and fake-bad instructions. Significant correlations between the nonconformity scale and certain PdX and Pd subscales were found only for the control group. Implications for test construction and for clinical interpretation under faking conditions are discussed.     

预测视角下双因子模型与高阶模型的模拟比较

双因子模型和高阶因子模型,作为既有全局因子又有局部因子的两个竞争模型,在研究中得到了广泛应用。本文采用Monte Carlo模拟方法,在模型拟合比较的基础上,比较了效标分别为外显变量和内潜变量时,两个模型在各种负荷水平下预测准确度的差异。结果发现,两种模型在拟合效果方面无显著差异;但在预测效度方面,当效标为显变量时,两个模型的结构系数估计值皆为无偏估计;而效标为潜变量时,高阶因子模型表现优于双因子模型:高阶因子模型的结构系数为无偏估计,双因子模型的结构系数估计值则在50%左右的情况下存在偏差。     

The interpretation of a test validity coefficient in terms of increased efficiency of a selected group of personnel

The predictive efficiency of a test used to select personnel is defined in terms of total effectiveness of the group thus selected, as compared with chance selection. The formula developed requires the use of an estimate of the ratio of average effectiveness of men selected to the average effectiveness of men not selected by the test. The predictive efficiency of the test varies directly with the magnitude of this ratio and also directly with the percentage rejected.     

Action-state orientation: construct validity of a revised measure and its relationship to work-related variables

Data from 6 studies (N = 945) were combined to examine the factor structure and construct validity of J. Kuhl's (1994a) measure of action-state orientation, the Action Control Scale (ACS). Initial confirmatory factor analysis (CFA) of 4 models showed poor fit with the data. Items that did not perform well were dropped, and the 4 CFA models were retested, resulting in support for a 3-factor solution corresponding to the Preoccupation, Hesitation, and Volatility subscales. Regression analyses demonstrated that the revised ACS subscales measure a construct distinct from cognitive ability and personality. The revised subscales showed differential statistically significant relationships with (a) self-regulatory and self-focus variables and (b) job attitudes and ratings of job performance. The authors conclude that action-state orientation is a construct relevant to applied settings and that future research should further examine this construct.     

The contribution of self-compassion in the relationship between social support and posttraumatic growth

IntroductionDespite well-established evidence on the relationship between social support and posttraumatic growth (PTG), there remains a paucity of evidence regarding the mechanism that underlie this relationship.ObjectiveThis study examined the relationship between social support and PTG, while exploring self-compassion as an intermediatory factor in this association.MethodA cross-sectional study with 447 college students was conducted in Trabzon, Turkey. The data were analyzed using Structural Equation Modelling (SEM).ResultsResults indicated that perceived social support and self-compassion were associated with PTG. Furthermore, a significant indirect effect between social support and PTG via self-compassion emerged.ConclusionBased on the study findings, tailored intervention programs targeting self-compassion and perceived social support in trauma-exposed young adults may be useful for promoting posttraumatic growth.     

The stability of validity coefficients over time: Ackerman's (1988) model and the general aptitude test battery

This study examined P. L. Ackerman's (1988) model of skill acquisition within an applied setting. Differences were examined between jobs on the basis of task consistency: changes in performance variability across experience, learning curves, and stability of ability-performance correlations across experience. Results showed the degree of task consistency influenced the shape of learning curves, with jobs composed of primarily consistent tasks improving more rapidly and reaching asymptote sooner. In addition, trends in ability-performance correlations were moderated by the degree of task consistency within ajob. Specifically, forjobs with primarily consistent tasks, general cognitive ability best predicted early performance whereas psychomotor ability best predicted later performance. In contrast, general cognitive ability was the strongest predictor across experience for jobs with primarily inconsistent tasks.