THE EFFECT OF CORRELATED ARTIFACTS AND TRUE VALIDITY ON THE ACCURACY OF PARAMETER ESTIMATION IN VALIDITY GENERALIZATION

The effect of non-zero intercorrelations among the three artifacts (criterion reliability, predictor reliability, and range restriction on the predictor) and true validity on the accuracy of estimation in two validity generalization models (Model 1and Model 2) was investigated. Only the Thylor Series Approximation 1(TSA1) procedure from Model 1 and the procedure with complete sample-based artifact data from Model 2 were included in this study. Six intercorrelation conditions (zero, low positive, medium positive, low negative, low negative and positive, and medium negative and positive) and two distributions of artifacts were used in this investigation. The two major conclusions from this study are: (a) Both models yielded reasonably accurate estimates of the mean of true validities (M_ρ) and (b) only Model 2 produced accurate estimates of the variance of true validities (V_ρ). The need for additional research for accurately estimating V_ρ when the three artifacts and true validity are correlated is recommended.     

EMPIRICALLY DERIVED JOB FAMILIES AS A FOUNDATION FOR THE STUDY OF VALIDITY GENERALIZATION

The purpose and background of a series of three studies dealing with validity generalization are discussed, an overview of the series is provided, and the initial study is described.
In the initial study a total of 76 insurance company jobs was analyzed by 203 raters in an effort to assess the potential usefulness of the Position Analysis Questionnaire (PAQ) as a job analysis device to be employed in a more extensive, companywide research program. Examination of rate-rerate and interrater reliability suggested that rate-rerate reliability was quite good, while interrater reliability estimates were somewhat lower. Cluster analysis techniques were applied to the five Overall and 17 Component dimensions of the PAQ yielding six job families in both analyses. The job families were described in terms of the various PAQ dimensions and were judged to be organizationally meaningful. Although the job clustering results were generally acceptable, it was conjectured that job families formed on the basis of company specific dimensions might prove even more meaningful.     

THE EFFECTS OF UNREPRESENTED STUDIES ON THE ROBUSTNESS OF VALIDITY GENERALIZATION RESULTS

Researchers conducting meta-analyses such as validity generalization can never be certain that their review contains all studies relevant to the research domain. Indeed, several authors in the past have noted ways in which research reviews may be systematically biased. A few techniques have emerged for addressing the issue of "missing studies" including the use of Rosenthal's (1979) file-drawer equation. Noting that Rosenthal's technique is inappropriate when applied to validity generalization findings, this paper develops a new method for assessing the vulnerability of validity generalization results to unrepresented or missing studies. The results of this new procedure are compared to the results of file-drawer analyses for 103 findings from validity generalization studies. We illustrate that this procedure more appropriately estimates the robustness of validity generalization results.     

VALIDITY GENERALIZATION: A REVIEW AND CRITIQUE OF THE CORRELATION MODEL

This paper presents a review and critique of the validity generalization research based on the correlation model. Emphasis was placed on integrating the applied validity generalization studies as well as the computer simulation studies which have tested the accuracy of the various validity generalization procedures in estimating the mean and variance of true validity coefficients. In general, this review indicated that the validity generalization procedures are fairly accurate in estimating the mean and variance of true validities, that the preponderance (e.g., 90%) of corrected validity coefficients for a single predictor-criterion relationship, based on a series of cumulative studies, when placed in distributional form are positive, and that a substantial proportion of observed validity coefficient variation can be attributed to statistical artifacts with sampling error accounting for the majority of the artifactual variance. Moreover, potential problems with validity generalization/meta analysis based on criterion-related validity coefficients were discussed. It was concluded that validity general-ization/meta-analysis is an important development in summarizing cumulative research, however, caution should be exercised in utilizing and interpreting the findings of such analyses. Directions for future research aimed at establishing trait-performance relationships were discussed.     

JOB CLASSIFICATION APPROACHES AND THE IMPLEMENTATION OF VALIDITY GENERALIZATION RESULTS

This paper compares two job classification methods for showing the appropriateness of cognitive tests in settings that were not involved in supplying data for a validity generalization analysis. One approach was an elaborate quantitative procedure that involved a lengthy job inventory and a multivariate item analysis. This approach was shown to be highly successful when applied to the responses from 1179 job inventories collected in 54 petroleum-petrochemical plants from 30 different companies. The other procedure involved simple job classification judgments by supervisors and incumbents. This latter approach was shown to be as effective, but was much less time consuming and costly. Professional and legal implications of these findings are discussed.     

FURTHER TESTS OF THE SCHMIDT-HUNTER BAYESIAN VALIDITY GENERALIZATION PROCEDURE1

Using an improved version of the Bayesian validity generalization model presented in Schmidt and Hunter (1977), this study showed that most of the between-study variation in observed validity coefficients of various aptitude and ability tests for two types of clerical work and for the job of first-line supervisor is artifactual in nature. These results cast additional doubt on the traditional belief that employment test validities are situationally specific. In addition, the results showed that generalization of validities to new settings was justified even where the hypothesis of situational specificity could not be rejected under the standards used in this study. Implications for the use of aptitude and ability tests in selection and for theory development in industrial-organizational psychology are discussed.     

ON THE VALIDITY OF THE RORSCHACH TEST

Rorschach protocols of 20 persons (normals, depressives, and schizophrenics) were analysed, using the inverted, or Q , method of factoring. Three factors emerged from the analysis and corresponded to the three psychiatric categories. Groups of items, allegedly measuring the isolated factors, were applied to a second population of 42 persons resulting in validity values of an encouraging magnitude. Finally, published protocols are cited, together with a list of discriminating items and a score matrix based on such items, so that further verification can be made by the reader.     

NOTE ON THE VALIDITY OF THE POWER LAW

On the basis of previous experimental findings, it is argued that the power law is not based on a true (subjective) ratio scale, but on a quasi-ratio scale. Hence, and in spite of the results of the cross-modality matching, the validity of the power law, as well as the validity of models and equations in which a subjective scale is involved, is doubted. It is further indicated that the validity of Brown's law can, under special conditions, be tested by the relation between subjective variables of velocity, length and time. Finally, the relation between the power law and Luce's theorem is discussed briefly.     

INTERNATIONAL VALIDITY GENERALIZATION OF GMA AND COGNITIVE ABILITIES: A EUROPEAN COMMUNITY META-ANALYSIS

This article reports on a series of meta-analyses into the criterion validity of general menatl ability (GMA) and specific cognitive ability tests for predicting job performance ratings and training success in the European Community (EC). Meta-analyses were computed on a large EC database examining the operational validity of GMA and other specific cognitive abilities, including verbal, numerical, spatial-mechanical, perceptual and memory ( N ranged from 946 to 16,065) across 10 EC member countries. The results showed that tests of GMA and specific cognitive ability are very good predictors of job performance and training success across the EC. Evidence for the international validity generalization of GMA and specific cognitive abilities was presented. The results for the EC meta-analyses showed a larger operational validity than previous meta-analyses in the U.S. for predicting job performance. For training success, the European and American results are very similar. Implications for the international generalizability of GMA test validities, practical use of cognitive ability tests for personnel selection, and directions for future research are discussed.     

GENERALIZATION OF NAMING RESPONSES TO OBJECTS IN THE NATURAL ENVIRONMENT AS A FUNCTION OF TRAINING STIMULUS MODALITY WITH RETARDED CHILDREN

Picture-cards, photographs, and real objects were compared as training stimuli in order to determine which best facilitated the generalization of naming responses learned in a special training room to real objects in the natural environments of four retarded children. The amount of transfer of naming behavior between the three stimulus modes and the average amount of training time required per stimulus mode were also assessed. Three of the four children displayed considerably more generalization to the real objects in the natural environment when they were trained with real objects. The fourth child displayed substantial generalization regardless of the training stimulus mode. No particular training stimulus mode clearly facilitated the transfer of naming responses to other modes or greatly reduced training time. The results of two supplementary procedures conducted with one child showed that: (1) training in several environments facilitated generalization to real objects in the natural environment when real objects were used as training stimuli but not when picture-cards were used, and (2) transfer from picture-cards to real objects was facilitated by training other picture-cards and the real objects portrayed by them at the same time.     

THE WHOLE BRAIN AS THE BASIS OR THE ANALOGICAL EXPRESSION OF GOD

Abstract. As human beings we inevitably try to explain our experience. In philosophical language, we deal with transcendent assertions and aspirations. The issue, then, is: how can we talk about what matters, given the structures inherent in language and basic to the way we are made? Instead of the philosophical category of Being, I advance a case for giving the human brain privileged status as an analogical expression of God, the symbol-concept of what matters most, and then suggest the illumination which can come with using that analogical expression, especially as that analogy connects us with the environment at the limbic level and constructs our world at the cerebral level.     

THE EFFECTS OF ECHOLALIA ON ACQUISITION AND GENERALIZATION OF RECEPTIVE LABELING IN AUTISTIC CHILDREN

This investigation, consisting of two experiments, was designed to assess the effects of autistic immediate echolalia on acquisition and generalization of receptive labeling tasks. Experiment 1 addressed whether autistic children could use their echolalia to facilitate acquisition. The results indicated that incorporating echolalia (echo of the requested object's label) into the task before manual response (handing the requested object to the experimenter) facilitated receptive labeling. Experiment 2 was designed to determine the effects of incorporating echolalia into task response on acquisition and subsequent generalization. These results indicated that echolalia facilitated generalization for echolalic autistic children but not for functionally mute autistic children. The results of the experiments are discussed in terms of stimulus control. Additionally, it is proposed that perhaps in certain cases, echolalia should not be eliminated, but used to advantage in receptive responding.     

ON THE VALIDITY OF SUBJECTIVE MEASURES OF COMPANY PERFORMANCE

Subjective measures of company performance are widely used in research and typically are interpreted as equivalent to objective measures. Yet, the assumption of equivalence is open to challenge. We compared the use of both types of measure in 3 separate samples. Findings were consistent in showing that: (a) subjective and objective measures of company performance were positively associated (convergent validity); (b) those relationships were stronger than those between measures of differing aspects of performance using the same method (discriminant validity); and (c) the relationships of subjective and objective company performance measures with a range of independent variables were equivalent (construct validity).     

ON THE METHODS AND THEORY OF CLUSTERING

The need for methods of clustering individuals into homogeneous groups seems clear. One hopes, by applying them to his data, to discover clusterings which may prove to be important. This aim appears straightforward, but the methods which exist do not necessarily satisfy them. The procedures which employ the correlation measure of profile similarity, and those which employ the distance measure are discussed. Technical and logical problems are shown to exist for both measures. The key defect in almost all clustering procedures seems to be the absence of a statistical model. The suggestion is made that the clustering problem be stated as a mixture problem. The need for further work by psychologists and statisticians is pointed out.