ON THE VALIDITY OF SUBJECTIVE MEASURES OF COMPANY PERFORMANCE

Subjective measures of company performance are widely used in research and typically are interpreted as equivalent to objective measures. Yet, the assumption of equivalence is open to challenge. We compared the use of both types of measure in 3 separate samples. Findings were consistent in showing that: (a) subjective and objective measures of company performance were positively associated (convergent validity); (b) those relationships were stronger than those between measures of differing aspects of performance using the same method (discriminant validity); and (c) the relationships of subjective and objective company performance measures with a range of independent variables were equivalent (construct validity).     

AN ASSESSMENT OF STUDENTS' CALIBRATION OF COMPREHENSION AND CALIBRATION OF PERFORMANCE USING MULTIPLE MEASURES

In the present study we examined students' ability to judge comprehension of text material (before actually taking a test) as well as ability to judge comprehension performance on text material (after taking a test). We examined each of these two abilities, termed calibration of comprehension and calibration of performance, by using multiple measures rather than just one as has traditionally been done in the literature. We also examined the role of students' self-assessed monitoring ability and self-image presentation in calibration skills. Results indicated that students were able to calibrate comprehension and performance although calibration levels were low, especially for calibration of comprehension. Students who calibrated comprehension well also tended to calibrate performance well. Several measures used to examine calibration of comprehension were consistent predictors ofperformance as were both quantitative and qualitative measures of calibration of performance. Students' calibration accuracy remained stable across measures and assessment conditions. Although students' self-assessed ability was not related to their actual calibration skills, self-image presentation was related to students'self-assessments and to theirability to calibrate performance. Implications and directions for future research are discussed.     

DETERMINATION AND EVALUATION OF RATE MEASUREMENTS IN THE ANALYSIS OF SPACE MEDICAL DATA

Measures of rate of change and of rate of rate of change are developed for application to physiological and psychological data. Examples are given of the use of these measures with heart-rate data for comparisons within and between subjects. It is shown that the measures provide information over and above that offered by means and variances, and that the measures appropriately reflect the impact of external variables.     

DEVELOPMENT OF MULTIPLE JOB PERFORMANCE MEASURES IN A REPRESENTATIVE SAMPLE OF JOBS

The goal of criterion development in Project A was to construct multiple measures of the major components of job performance such that the total performance domain for a representative sample of the population of entry-level enlisted positions in the U.S. Army was covered. These measures were to be used as criteria against which to validate both experimental and existing predictors of job performance. The initial model specified that performance is multidimensional within two major categories of dimensions designated as organization-wide and job specific. The development strategy involved describing the total domain of job content via extensive task analyses and critical incident analyses, generating the critical performance dimensions that constitute it, constructing measures for each dimension, and evaluating each measure using expert judgment and field test data. The specific measures developed consisted of rating scales, tests of job knowledge, hands-on job samples, and archival records. The major steps in the job analyses, content sampling, instrument construction, and instrument evaluation are described, and the final array of criterion measures is presented.     

ANALYSIS OF THE CONSISTENCY OF OBJECTIVE MEASURES OF SEXUAL AROUSAL IN WOMEN

Eight adult women volunteers viewed the same erotic film in two different sessions. Their resulting genital responses were recorded simultaneously by three different measures: vaginal pressure pulse, vaginal blood volume, and labial temperature change. During the erotic stimulation, the vaginal pressure pulse and labial responses always increased, and the vaginal blood volume response increased on all but two occasions. Although there was considerable intersubject variability in each genital measure, all three measures were found to have some intrasubject consistency over sessions with respect to either their response amplitudes or patterns, with labial temperature being the most consistent on both parameters. The relationship between the response patterns of the three measures during the film was also relatively consistent across sessions, as was the correspondence between subjective ratings of arousal and both vaginal pressure pulse and labial responses but not vaginal blood volume response. To overcome the problem of considerable intrasubject variability of response amplitudes, it was suggested that the inclusion in the data analysis of several parameters of response patterns, which were relatively stable over sessions, might facilitate the evaluation of a treatment.     

JUDGMENTS OF SEXISM: A COMPARISON OF THE SUBTLETY OF SEXISM MEASURES AND SOURCES OF VARIABILITY IN JUDGMENTS OF SEXISM

We compared the subtlety of four measures of sexism and sources of variation in male and female psychology students' judgments that beliefs from these scales and everyday behaviors were sexist. Participants judged traditional gender role and hostile sexist beliefs as more sexist than benevolent and modern sexist beliefs, indicating the latter were more subtle measures of sexism. Participants also judged traditional gender role behaviors as more sexist than unwanted sexual attention, suggesting the latter may less readily be identified as sexist. Variation in judgments of beliefs as sexist was related to differences in likelihood of endorsing such beliefs. This relation fully accounted for the tendency for men to be less likely to judge beliefs as sexist in comparison to women. Endorsement of Modern and Hostile Sexist beliefs was related to judgments of behaviors as sexist. The implications of the results for scale usage and identifying sexist behavior are discussed.     

当前阅读研究中眼动指标述评

文章重点介绍了阅读研究中常用的眼动指标,它们包括注视时间、眼跳、回视和瞳孔直径等,从字和区域这两个方面探讨了这些眼动指标所反映的认知加工过程,以及选择这些指标时应该注意的问题。最后对眼动指标的选择和使用等问题提出了几点看法。     

CONSTRUCT VALIDITY OF TWO CATEGORIES OF ASSESSMENT CENTER DIMENSION RATINGS

The construct validity of assessment center final dimension ratings was examined within a nomological network of cognitive and personality measures. Four hundred forty-one employees of a large mid-western petroleum company were assessed on 11 dimensions in two broad categories and completed four tests. Results showed that several cognitive ability measures related more strongly to performance-style dimension ratings than to interpersonal-style dimension ratings, providing evidence for convergent and discriminant validity. Correlation analysis and factor analysis support the two a priori interpersonal- and performance-style categories. The results suggest that final dimension ratings possess construct validity and that assessors can differentiate between two broad categories of assessment dimensions.     

A META-ANALYSIS OF THE RELATIONS AMONG TRAINING CRITERIA

An augmented framework for training criteria based on Kirkpatrick's (1959a, 1959b, 1960a, 1960b) model divides training reactions into affective and utility reactions, and learning into post-training measures of learning, retention, and behavior/skill demonstration. A total of 34 studies yielding 115 correlations were analyzed meta-analytically. Results included substantial reliabilities across training criteria and reasonable convergence among subdivisions of criteria within a larger level. Utility-type reaction measures were more strongly related to learning or on-the-job performance (transfer) than affective-type reaction measures. Moreover, utility-type reaction measures were stronger correlates of transfer than were measures of immediate or retained learning. These latter findings support recent concurrent thinking regarding use of reactions in training (e.g., Warr & Bunce, 1995). Implications for choosing and developing training criteria are discussed.     

IN SUPPORT OF PERSONALITY ASSESSMENT IN ORGANIZATIONAL SETTINGS

Personality constructs have been demonstrated to be useful for explaining and predicting attitudes, behaviors, performance, and outcomes in organizational settings. Many professionally developed measures of personality constructs display useful levels of criterion-related validity for job performance and its facets. In this response to Morgeson et al. (2007) , we comprehensively summarize previously published meta-analyses on (a) the optimal and unit-weighted multiple correlations between the Big Five personality dimensions and behaviors in organizations, including job performance; (b) generalizable bivariate relationships of Conscientiousness and its facets (e.g., achievement orientation, dependability, cautiousness) with job performance constructs; (c) the validity of compound personality measures; and (d) the incremental validity of personality measures over cognitive ability. Hundreds of primary studies and dozens of meta-analyses conducted and published since the mid 1980s indicate strong support for using personality measures in staffing decisions. Moreover, there is little evidence that response distortion among job applicants ruins the psychometric properties, including criterion-related validity, of personality measures. We also provide a brief evaluation of the merits of alternatives that have been offered in place of traditional self-report personality measures for organizational decision making. Given the cumulative data, writing off the whole domain of individual differences in personality or all self-report measures of personality from personnel selection and organizational decision making is counterproductive for the science and practice of I-O psychology.     

ABSTRACTS OF LITERATURE

The importance of diagnosis in family treatment is emphasized. Diagnostic techniques include not only gathering historical information but also employing action-oriented measures such as family sculpting.     

TOWARDS AN UNDERSTANDING OF INTEGRITY TEST SIMILARITIES AND DIFFERENCES: AN ITEM-LEVEL ANALYSIS OF SEVEN TESTS

A judgmental sort of 798 items from 7 paper-and-pencil integrity tests produced 23 thematic composites. Patterns of correlations between these composites and the 7 integrity measures and 2 Big Five measures shed light on similarities and differences between different integrity tests. Principal components analysis of 23 composites indicated 4 principal components that further illuminate the content domain of integrity tests. The relationships between 4 integrity principal components and integrity test scores as well as measures of the Big Five dimensions of personality are reported. The findings suggest that integrity tests can differ in their emphasis on various thematic composites, and, yet, be very similar in terms of their standing on the 4 integrity principal components. Different integrity tests can be quite different in terms of surface content, and, yet, assess the same underlying constructs.     

QUANTITATIVE MEASURES OF CONSTRUING IN CHILD ABUSE SURVIVORS

Quantitative aspects of construing associated with long-term effects of abusive childhood experiences are examined within a personal construct theory framework. Indices of negative self-construing, reduced identification with others, and extreme construing are compared from the elicited Repertory Grids (RepGrids) of survivors of sexual, physical, and emotional abuse and nonabused controls (N = 336). Responses to the Automatic Thoughts Questionnaire (ATQ) are also compared, as a further measure of negative self-constructions. Abuse histories are particularly associated with RepGrid and ATQ measures of negative self-constructions and with RepGrid measures of perceived differences between self and parents. Among cognitive and construct measures, the ATQ is most strongly related to standardized measures of symptom distress and social maladjustment, but RepGrid measures also make a unique contribution. From the RepGrid, extreme perceptions of self are related to symptom distress and social maladjustment, and self-ideal discrepancy is related to social maladjustment. Shared method variance and standardized statements specifically relevant to self-evaluation and pathology may contribute to the predictiveness of the ATQ. In contrast, the RepGrid samples a more general range of idiographic constructs used in comparing people.     

RELAXATION TREATMENT OF PSEUDOINSOMNIA AND IDIOPATHIC INSOMNIA: AN ELECTROENCEPHALOGRAPHIC EVALUATION

Twenty-nine insomniacs underwent four consecutive sleep laboratory evaluations before and after receiving tension-release relaxation training, no-tension-release relaxation training, or no-treatment. On the basis of the discrepancy between subjective and EEG-defined measures of latency to sleep onset, subjects were classified as pseudoinsomniacs or idiopathic insomniacs. As predicted, tension-release relaxation was significantly more effective than the other two conditions on subjective sleep measures, regardless of insomnia subtype and on objective sleep measures only for idiopathic insomniacs. Subjective improvement was maintained at 12-month followup. Numerous differences between the two subtypes emerged on pretherapy and during-therapy measures distinct from the latency measures, but changes on those variables were unrelated to outcome improvement.     

A REVIEW AND EVALUATION OF THE PSYCHOMETRIC PROPERTIES OF THE JOB DIAGNOSTIC SURVEY

Empirical research is reviewed to evaluate the test-retest reliability, internal consistency, scale discrimination, factorial validity, convergent validity across raters and methods, and methods bias of the Job Diagnostic Survey (JDS) of Hackman and Oldham (1975, 1980). The review shows that the JDS has important psychometric limitations, but is able–when used properly–to provide useful information about perceived job properties. Suggestions are made for improving the JDS and for developing additional instruments that assess a broader array of job dimensions and that assess them more objectively than current measures. Job characteristics researchers need a diverse portfolio of measures to accomplish different purposes.     

CONCEPTUAL INTEGRATION AND EMPIRICAL TEST OF JOB DESIGN AND COMPENSATION RELATIONSHIPS

A conceptual integration of job design and compensation draws on interdisciplinary job design, job evaluation, and labor economic theory. It is argued that job design influences the number and level of skills required and the degree to which jobs are physically aversive or hazardous. External labor markets also respond to skill and physical requirements. Job evaluation links job design and market forces by analyzing jobs'compensable factors that reflect these requirements, and then relating them to the market through wage surveys across firms. An empirical examination presents relationships between job design and pay or job evaluation measures. Strongly supportive results replicate in two separate samples (total n = 213 jobs) which differ in industries, job types, skill levels, job design measures, job evaluation measures, and labor markets. Motivational job design had higher job evaluation measures reflecting higher skill requirements, and mechanistic and perceptual/motor design had lower evaluation measures reflecting lower skill requirements. Biological design had lower evaluation measures reflecting physical requirements.     

MEASURING SOCIAL-COMMUNICATIVE ANXIETY: SOCIAL DESIRABILITY AND THE FAKABILITY OF SCALE RESPONSES

Three studies are reported that examine the social desirability and fakability (transparency) of a number of self-report measures of social-communicative anxiety. Results indicate that social desirability, assessed as a personality dimension, is generally unrelated to measures of social and communicative anxiety. However, virtually all measures of the anxiety were transparently fakable. When asked to portray themselves positively, subjects consistently responded as low-anxious; when asked to look undesirable, subjects completed the measures to suggest high anxiety.     

TWO MEASURES OF PREFERENCE DURING FORCED-CHOICE ASSESSMENTS

A forced-choice preference assessment was conducted in which two dependent measures were used to select preferred stimuli: (a) approach responding and (b) latency to the first aberrant response. Stimuli identified as preferred based on both dependent measures were then evaluated during treatment. The results suggested that latency may be a useful measure in the selection of preferred stimuli during forced-choice assessments.     

COMPARATIVE STUDIES OF SENSORY RELATEDNESS OF ODOURS

Measures of relatedness of odours were obtained by means of two psychophysical methods and one electrophysiological technique. The psychophysical data consisted of similarity judgments and confusion measures. The electrophysiological data were based upon statistical treatment of the responses of single units in the olfactory bulb of the frog. Significant rank correlations were obtained between the results of the three methods. The significance of psychological relatedness data for the study of basic receptor processes is discussed.     

ON THE INTERCHANGEABILITY OF OBJECTIVE AND SUBJECTIVE MEASURES OF EMPLOYEE PERFORMANCE: A META-ANALYSIS

A meta-analysis of studies containing both objective and subjective ratings of employee performance resulted in a corrected mean correlation of .389. This value, although significantly greater than zero, indicates that objective and subjective performance measures should not be used interchangeably. Moreover, in no moderator subgroup examined did the correlation suggest convergent validity. After discussing issues related to resolving the previous anomalies of primary and meta-analytic results, a secondary analysis suggested that objective and subjective measures of the same construct at the same level may be used inter-changeably. The secondary analysis, however, was based on a very limited sample. Future research should address the appropriate dimensionality of employee performance.