Methodological pitfalls in predicting counseling success

Prediction models employing multiple linear regression of raw scores, multiple linear regression of factor scores, the single best predictor, and a nine-point decision rule index were compared. The subjects were 296 clients undergoing vocational counseling and evaluation. Predictor variables included performance ratings, demographic variables, and WAIS subtest scores; the criterion was employment status upon program completion. The least statistically sophisticated model, employing the single best predictor, was the most successful approach. Considerable shrinkage in power of prediction was demonstrated upon cross-validation particularly for multiple linear regression of raw scores model, indicating the necessity of cross-validating prediction schemes. Additional suggestions are made to those designing prediction studies.     

The prediction of intelligence in preschool children using alternative models to regression

Statistical prediction of an outcome variable using multiple independent variables is a common practice in the social and behavioral sciences. For example, neuropsychologists are sometimes called upon to provide predictions of preinjury cognitive functioning for individuals who have suffered a traumatic brain injury. Typically, these predictions are made using standard multiple linear regression models with several demographic variables (e.g., gender, ethnicity, education level) as predictors. Prior research has shown conflicting evidence regarding the ability of such models to provide accurate predictions of outcome variables such as full-scale intelligence (FSIQ) test scores. The present study had two goals: (1) to demonstrate the utility of a set of alternative prediction methods that have been applied extensively in the natural sciences and business but have not been frequently explored in the social sciences and (2) to develop models that can be used to predict premorbid cognitive functioning in preschool children. Predictions of Stanford–Binet 5 FSIQ scores for preschool-aged children is used to compare the performance of a multiple regression model with several of these alternative methods. Results demonstrate that classification and regression treesprovided more accurate predictions of FSIQ scores than does the more traditional regression approach. Implications of these results are discussed.     

JOB SATISFACTION, BEHAVIORAL INTENTION, AND ABSENTEEISM AS PREDICTORS OF TURNOVER

Measures of overall job satisfaction, intention about remaining with the company, and frequency of absences were correlated with termination over two one-year periods for a sample of female clerical employees. A step-wise multiple regression indicated that both intent to remain with the company and frequency of absences added to the prediction of turnover during both one-year periods. Job satisfaction did not add to the prediction for either year.     

The Relation of Cognitive and Memorial Processes to Persuasion in a Simulated Jury Trial

Four experiments investigated the dependence of persuasion on cognitive factors. All experiments employed a court case for which 795 subjects acted as jury members, reading summaries of both the prosecution and defense's testimony. The amount of objective information on both sides of the case was varied. Persuasion was a position function of the number of prosecution arguments and the number of defense arguments. This finding was extended by obtaining measures of the subjects'cognitive reactions to the case as well as their opinions and by following both of these measures over time. Both analysis of variance and multiple regression techniques showed that subjects could have derived their opinions from their cognitions about the case. This relationship also held up over time. These results suggest the general form of an information-processing theory of persuasion. One prediction of this theory is for an asymptotic function relating objective information to persuasion. This prediction received empirical support.     

TESTING FOR FAIRNESS WITH A MODERATED MULTIPLE REGRESSION STRATEGY: AN ALTERNATIVE TO DIFFERENTIAL ANALYSIS1

It is argued that analyses of subgroup differences utilizing a bivariate correlation strategy do not provide an adequate examination of test fairness. An analysis of differential prediction, which involves slopes and intercepts of regression lines results in more complete coverage of the test fairness issue, since the overall regression line determines the way in which a test is used for prediction. While subgroup correlation coefficients yield information concerning the slopes and intercepts, means and standard deviations must also be examined. A moderated multiple regression strategy is recommended as an alternative to separate analyses by subgroups. An ordered step-up regression procedure is presented which is more encompassing than the bivariate strategies, while avoiding inherent problems associated with subgroup coding in multiple regression.     

Life history correlates of ministerial success

Life history or biodata correlates of ministerial success were investigated for a group of 92 Seventh-Day Adventist ministers. The criterion data reflected their current level of success and the predictors reflected their status as seminarians between the years 1940 to 1950. Two significant bivariate correlations indicated that successful ministers chose their career later than less successful ones and that earning college expenses was predictive of success. A stepwise multiple regression process was used to develop a weighted prediction model, but the model did not hold when cross-validated.     

Theory of mind and executive function: working-memory capacity and inhibitory control as predictors of false-belief task performance

This study of the relationship between theory of mind and executive function examined whether on the false-belief task age differences between 3 and 5 ears of age are related to development of working-memory capacity and inhibitory processes. 72 children completed tasks measuring false belief, working memory, and inhibition. Significant age effects were observed for false-belief and working-memory performance, as well as for the false-alarm and perseveration measures of inhibition. A simultaneous multiple linear regression specified the contribution of age, inhibition, and working memory to the prediction of false-belief performance. This model was significant, explaining a total of 36% of the variance. To examine the independent contributions of the working-memory and inhibition variables, after controlling for age, two hierarchical multiple linear regressions were conducted. These multiple regression analyses indicate that working memory and inhibition make small, overlapping contributions to false-belief performance after accounting for age, but that working memory, as measured in this study, is a somewhat better predictor of false-belief understanding than is inhibition.     

Multiple bi-serial and multiple point bi-serial correlation

Normal equations, using data in various forms, are presented for securing the regression weights for prediction of a dichotomized criterion, and a simplified equation for the estimation of the multiple bi-serial or multiple point bi-serial, depending upon the proper assumption as to the nature of the distribution of the criterion, on the basis of these maximal weights is given also. The weights, unaffected by the assumption as to the nature of the criterion, are identical (or proportional) to those found by the discriminant function approach based upon analysis of variance. The author holds that the present multiple correlation approach is both easier and more informative than the discriminant function (analysis of variance) approach and suggests that the discriminant function be abandoned in favor of multiple bi-serial and/or multiple point bi-serial correlation and regression.     

The psychology of moderate prediction : I. Experience with multiple determination

In the experiments reported here, individuals with experience in a multivariate prediction setting showed considerable moderation of subsequent univariate predictions, compared to those without such experience. We show that such moderation of prediction does not result from an abstract rule of regression to the mean; rather, it can be explained by the named error model. According to this model, missing predictors are treated as an error term, with their unknown values replaced by central tendencies. Experiment 1 demonstrates the phenomenon of moderation following multivariate experience and explores its generalization to novel predictors. Moderation occurs even for a perfectly valid predictor, contrary to normative application of a regression strategy. Experiment 2 shows that the phenomenon depends on lack of correlation among the multivariate predictors. This accords with the named error model, which asserts that if missing predictors are perceived to be correlated with the available predictor, their unknown values are replaced by extreme values rather than by central tendencies. Experiment 3 shows that mere exposure to additional predictors has no effect; experience in which multiple predictors are used to make numerical predictions seems to be necessary in order to obtain subsequent moderation. In Experiment 4, feedback is introduced. Moderation of prediction results even without prior multivariate experience. However, multivariate experience produces the moderation effect much more quickly.     

Job demands, job resources and individual innovation at work: going beyond Karasek's model?

The job demands-control model is one of the most recognized models in occupational stress research. It has, however, provided contradictory results, and the active learning hypothesis derived from this model has been under-researched in comparison with research on the stress hypothesis. The main aim of this study is to test the Job Demands Resources Model in the prediction of individual innovation at work as an active coping strategy. Results with hierarchical multiple regression analyses provide empirical support for this model. We found a positive relationship between job demands and individual innovation in situations characterized by high job resources. Finally, we discuss the limitations and practical implications of this study.     

Multivariate approaches to the identification of delinquency proneness in adolescent males

A sample of 337 adolescent male students were surveyed for demographic, individual, school, and familial functioning and delinquency status to investigate two questions relevant to prediction of adolescent delinquency proneness. First, three methods of scoring a delinquency self-report measure (frequency, variety, and seriousness) were compared to assess their differential relevance to the prediction of delinquency proneness. Second, a multivariate model was examined to assess its explanatory ability for identification of delinquency proneness. Findings, replicated through a series of regression analyses, demonstrate that age of onset is the best predictor. Other than family functioning, psychosocial indicators add little to the predictive model. Third, a specific factor model was preferable to a "risk count" method. Finally, the advantage of self-reports of delinquent behavior over official records is discussed as is the comparability of self-report scoring procedures. How self-reported delinquency is scored is not as critical as previously thought.     

基于两水平回归模型的调节效应分析及其效应量

使用多元回归法进行调节效应分析在社科领域已常有应用。简述了目前多元回归法的调节效应分析存在的不足,包括人为变换检验模型、自变量和调节变量区分不足、误差方差齐性的假设难以满足、调节效应量指标ΔR²没有直接测量调节变量对自变量与因变量关系的调节程度。比较好的方法是用两水平回归模型进行调节效应分析并使用相应的效应量指标。在介绍新方法和新效应量后,总结出一套调节效应的分析流程,通过一个例子来演示如何用Mplus软件进行两水平回归模型的调节效应及其效应量分析。最后讨论了两水平回归模型的调节效应分析的发展,包括稳健的调节效应分析、潜变量的调节效应分析、有调节的中介效应分析和有中介的调节效应分析等。     

When is a diagnosis worth making? A statistical comparison of two prediction strategies

For a setting in which the sole goal is to predict a criterion accurately, two strategies are compared, (1) multiple linear regression directly from a set of predictors and (2) using predictors to diagnose individuals and then using only the diagnosis to predict the criterion. These are mathematical models of two common methods for making clinical predictions. This article derives equations for each statistical strategy's validity (and a formula comparing them, for a special case). Prediction accuracies are compared over a simplified but broad parameter space and four conclusions follow. Changing disorder prevalences (in the range 0.1 to 0.5) have small effect on the relative preferability of the two strategies; the effect of varying prevalence differs according to population separation on predictors and criterion. As within-population covariances of predictors with criterion decline below zero (assuming variables are scaled so that the less common population scores higher on predictors and criterion), diagnoses become increasingly preferable as a prediction strategy; as they rise above zero the opposite trend is observed. As populations become better separated on predictors or criterion, diagnoses compare more favorably. Most importantly, over almost all of the parameter space which one would expect to encounter in clinical psychology and psychiatry, multiple linear regression is much superior.     

Toward a theory of individual differences and leadership: understanding the motivation to lead.

A broad, integrative theoretical framework for understanding the relationship between individual differences and various leader behaviors is presented; it proposes a new individual-differences construct called the motivation to lead (MTL). A large-scale study using 3 samples in different occupational and cultural contexts shows 3 factors underlying MTL, namely, affective-identity, noncalculative, and social-normative MTL. A parsimonious model of antecedents to MTL is developed through hierarchical regression modeling and is cross-validated using confirmatory latent variable modeling. MTL is shown to provide incremental validity over other predictors such as general cognitive ability, values, personality, and attitudes in the prediction of 2 behavioral measures of leadership potential. Findings are discussed with reference to the theoretical framework proposed for understanding individual differences in leader behavior.     

Testing main effects and interactions in latent curve analysis

A key strength of latent curve analysis (LCA) is the ability to model individual variability in rates of change as a function of 1 or more explanatory variables. The measurement of time plays a critical role because the explanatory variables multiplicatively interact with time in the prediction of the repeated measures. However, this interaction is not typically capitalized on in LCA because the measure of time is rather subtly incorporated via the factor loading matrix. The authors' goal is to demonstrate both analytically and empirically that classic techniques for probing interactions in multiple regression can be generalized to LCA. A worked example is presented, and the use of these techniques is recommended whenever estimating conditional LCAs in practice.     

The reliability of the Thematic Apperception Test

Controversy over the TAT's reliability may stem largely from the mis-application of traditional psychometric measures, which are inappropriate to this test. The TAT is implicitly based on a multiple regression model, for which coefficient alpha is not appropriate. Also, test-retest correlations may be adversely affected by the standard instructions to write a "creative" story. In a test-retest study, 47 high school students retook the TAT after a year with instructions designed to break the implicit set to produce a new and different story from that previously written. The test-retest correlations were r = .48 (need for affiliation) and .56 (need for intimacy), or approximately the same as those for, e.g., the MMPI, 16PF, and CPI, It was demonstrated that this high stability over time was not due to subjects' recalling and repeating previous responses. Finally, it was shown that alpha considerably underestimated the test-retest reliability, contrary to assumptions of classical psychometrics.     

Perceived trustworthiness of knowledge sources: the moderating impact of relationship length

Prior meta-analytic evidence has indicated no association between relationship length and perceived trustworthiness. Viewing trustors as information processors, the authors propose a model in which relationship length, although having no direct effect on perceived trustworthiness, moderates the association between perceived trustworthiness and the basis on which people decide to trust each other. Specifically, as trustors learn about others, they base their trust on different kinds of information (demographic similarity, trustworthy behavior, and shared perspective). Hierarchical multiple regression analyses of a field survey of supervisors and subordinates from 3 companies (N = 88) provide evidence consistent with this prediction: Perceived trustworthiness is associated with demographic similarity in newer relationships, with trustworthy behavior in relationships that are neither brand new nor old but in-between, and with shared perspective in older relationships.     

A multiple-observations model for response latency and the latencies of correct and incorrect responses in recognition memory

A model for response latency in recognition memory is described which is a strength model incorporating the notion of multiple observations and with the additional assumptions that the variance of the strength distributions increase with set size and that the observer attempts to keep his error rate at a constant level over set size. It is shown that the model can, without recourse to particular parameter values, predict a near linear RT set-size function and, since it is a (TSD) model in its decision aspects, can account for errors and hence error latencies in the recognition task. After the model is described, two experiments are performed which test the prediction that correct mean latency is generally shorter than incorrect mean latency. The prediction is confirmed and this feature is discussed in general, the model being compared with that of Juola, Fischler, Wood, and Atkinson (1971) in this respect. Some possible modifications to the latter model are also considered.     

Policy Capturing with Ridge Regression

A policy capturing method combining human judgment with ridge regression is offered which results in superior judgment policy models. The new method (termed smart ridge regression) was tested against four others in seven judgment policy capturing applications. Performance criteria were two cross-validation indices: cross-validated multiple correlation and mean squared error of prediction of new judgments. Smart ridge regression was found to outperform ordinary least squares regression and conventional ridge regression, as well as subjective weighting and equal weighting of cues.