The sentence-verification paradigm: A case study of two conflicting aproaches to individual differences

The contrasting approaches of differential psychology and cognitive psychology to the same individual differences data are outlined. Using illustrative data from the Clark and Chase (1972) sentence-picture verification task, four loci of conflict between these two disciplines are identified. These areas of conflict center around issues of (1) theory versus measurement, (2) meaningfulness versus reliability, (3) linearity of relationships, and (4) discontinuities in performance. We conclude on the basis of observed incompatibilities that a simple derivation of differential psychology from cognitive psychology is not likely, but separate development of complementary theories may be possible.     

The Negotiation Matching Process: Relationships and Partner Selection

We present 3 studies that examine the process of partner selection in negotiations and the influence that relationships may have on the partner-selection decision. In Study 1, we present a simulated matching market experiment in which we compare the matching process when relationships can influence the partner-selection decision with the matching process when relationships cannot influence this decision. We find that when relationships are not allowed to influence the matching process, there are more economically optimal agreements, a larger market surplus, and more search activity. In Study 2, a simulation of a naturally occurring market selection process provides additional quantitative support for the findings from Study 1 and offers qualitative data on the reasoning process behind partner selection, including the social factors that are influential in this decision. Study 3 utilizes a repeated trial experimental simulation to offer further support for the negative relationship between relationships and economic outcomes and to suggest that the relative power of the negotiators influences the extent to which relationships help versus hurt individual profitability. Implications of these findings are discussed.     

Frontal tests and models for cognitive ageing

Results of investigations of how specific behavioural changes in old age may be related to local changes in the brain have been discrepant because of neglect of methodological issues. Among these are neglect of the reliability and validity of frontal and executive tests; the operational definition and construct validity of hypothetical functional processes such as “inhibition”; the nature and origins of mutual relationships between tests; the relationships of test performance to higher order constructs of general mental ability such as “gf” and, most importantly, problems of cohort selection arising from imprecise definitions of the models of the ageing process. These problems are illustrated by a discussion of the results of several experiments in which younger and older groups of people are compared on frontal and executive tests.     

A JOB SAMPLING APPROACH TO MERIT SYSTEM EXAMINING1

The application of content validity to a merit examining process or rating schedule requires an extension of the concept beyond that of work samples and tests of knowledges or skills to measures of ability and personal characteristics. A method for accomplishing this without violating the principles of content validity is presented. This technique, called the job sampling approach, is a task-based, structured system of eliciting the information necessary to construct the rating schedule from sources most able to provide that information and for using the information to construct the rating schedule and linking it to job performance. The steps include: definition of the performance domain of the job in terms of process statements; identification of the selection and measurement objectives of the organization; development of the measurement domain in relation to the performance domain and to the selection and measurement objectives; and demonstration that a close match between the performance domain and the measurement domain was in fact achieved.     

MEASUREMENT ERROR IN RESEARCH ON HUMAN RESOURCES and FIRM PERFORMANCE: HOW MUCH ERROR IS THERE AND HOW DOES IT INFLUENCE EFFECT SIZE ESTIMATES?

Studies of the relationship between human resource (HR) practices and firm performance typically use a single respondent to assess firm level HR practices or HR effectiveness. However, previous research in other substantive areas suggests that rater differences are a potentially important source of measurement error. We demonstrate analytically the potential consequences of both random and systematic measurement error in research on HR and firm performance. However, our main focus is on random error and we show how generalizability theory can be applied to obtain better estimates of reliability by simultaneously recognizing multiple sources (e.g., items, raters) of random measurement error. These more inclusive reliability estimates, in turn, offer the possibility of more precisely quantifying substantive relationships in the HR and firm performance literature. In our sample, reliabilities (as estimated by generalizability coefficients) for single-rater assessments of HR variables were generally below .50. This degree of measurement error, if present in substantive studies on HR and firm performance, could lead to considerable bias, given that an unstandardized regression coefficient is corrected for measurement error in the independent variable by dividing by its reliability coefficient (not its square root). We also found only limited convergent validity between HR and line managers ratings of a second type of HR measure, HR effectiveness. In general, our findings suggest that future researchers need to devote greater attention to measurement error and construct validity issues. Our study provides an example of how generalizability theory can be useful in this pursuit.     

儿童早期空间非标准测量能力发展研究进展

测量是数学的主题之一。空间非标准测量能力的发展对儿童数学认知能力的发展有重要作用,对儿童思维抽象性、逻辑性、创造性的培养,问题解决能力的提高有重要影响。文章通过对近四十年来儿童空间非标准测量能力发展研究的成果进行概括分析,发现研究者主要对儿童空间非标准测量的认知加工线索,长度测量策略与工具的使用,空间非标准测量知识的获得顺序以及相关的测量教学顺序,空间非标准测量能力与守恒和推理能力的关系等问题进行了研究。并在此基础上提出了儿童空间非标准测量能力发展研究需进一步探讨的问题。     

Re-examining the Relationship Between Need for Cognition and Creativity: Predicting Creative Problem Solving Across Multiple Domains

Prior studies have demonstrated inconsistent findings with regard to the relationship between need for cognition and creativity. In our study, measurement issues were explored as a potential source of these inconsistencies. Structural equation modeling techniques were used to examine the factor structure underlying the 18-item need for cognition scale in three independent samples drawn from three prior studies. In addition, the relationships between need for cognition and the quality, originality, and elegance of creative problem solutions across multiple domains of performance were analyzed. Across all 3 samples, the bifactor model fit the data better than the 2-factor and 1-factor need for cognition models. After controlling for method-specific variance, the trait factor of the bifactor model showed consistently positive relationships with all 3 facets of creative performance and showed stronger relationships with these facets than the composite score results reported in prior studies. Implications for research and practice are discussed.     

Examining the item response process to personality measures in high-stakes situations: Issues of measurement validity and predictive validity

We conducted two experimental studies with between-subjects and within-subjects designs to investigate the item response process for personality measures administered in high- versus low-stakes situations. Apart from assessing measurement validity of the item response process, we examined predictive validity; that is, whether or not different response models entail differential selection outcomes. We found that ideal point response models fit slightly better than dominance response models across high- versus low-stakes situations in both studies. Additionally, fitting ideal point models to the data led to fewer items displaying differential item functioning compared to fitting dominance models. We also identified several items that functioned as intermediate items in both the faking and honest conditions when ideal point models were fitted, suggesting that ideal point model is “theoretically” more suitable across these contexts for personality inventories. However, the use of different response models (dominance vs. ideal point) did not have any substantial impact on the validity of personality measures in high-stakes situations, or the effectiveness of selection decisions such as mean performance or percent of fakers selected. These findings are significant in that although prior research supports the importance and use of ideal point models for measuring personality, we find that in the case of personality faking, though ideal point models seem to have slightly better measurement validity, the use of dominance models may be adequate with no loss to predictive validity.     

FAKING IN PERSONNEL SELECTION: TRADEOFFS IN PERFORMANCE VERSUS FAIRNESS RESULTING FROM TWO CUT-SCORE STRATEGIES

Most faking research has examined the use of personality measures when using top-down selection. We used simulation to examine the use of personality measures in selection systems using cut scores and outlined a number of issues unique to these situations. In particular, we compared the use of 2 methods of setting cut scores on personality measures: applicant-data-derived (ADD) and nonapplicant-data-derived (NADD) cut-score strategies. We demonstrated that the ADD strategy maximized mean performance resulting from the selection system in the face of applicant faking but that this strategy also resulted in the displacement of deserving applicants by fakers (which has fairness implications). On the other hand, the NADD strategy minimized displacement of deserving applicants but at the cost of some mean performance. Therefore, the use of the ADD versus NADD strategies can be viewed as a strategic decision to be made by the organization, as there is a tradeoff between the 2 strategies in effects on performance versus fairness to applicants. We quantitatively outlined these tradeoffs at various selection ratios, levels of validity, and amounts of faking in the applicant pool.     

Experience in integrative negotiations: What needs to be learned?

In this research we focus on the roles of experience and understanding in fostering integrative negotiation performance. We report on two experiments in which we distinguish between understanding opponents’ general priorities among issues versus understanding their specific gains for particular offers. Although experience enhanced integrative performance even in the absence of understanding, we found that understanding the specific gains had an incremental effect on performance. We conclude that while generally acknowledging opponents’ interests is not sufficient, the additional inferential step of assessing their specific gains throughout the negotiation process is advantageous.     

Collaborative Measurement Development as a Tool in CBPR: Measurement Development and Adaptation within the Cultures of Communities

This paper describes the processes we engaged into develop a measurement protocol used to assess the outcomes in a community based suicide and alcohol abuse prevention project with two Alaska Native communities. While the literature on community-based participatory research (CBPR) is substantial regarding the importance of collaborations, few studies have reported on this collaboration in the process of developing measures to assess CBPR projects. We first tell a story of the processes around the standard issues of doing cross-cultural work on measurement development related to areas of equivalence. A second story is provided that highlights how community differences within the same cultural group can affect both the process and content of culturally relevant measurement selection, adaptation, and development.     

Methodological Challenges in Causal Research on Racial and Ethnic Patterns of Cognitive Trajectories: Measurement, Selection, and Bias

Research focused on understanding how and why cognitive trajectories differ across racial and ethnic groups can be compromised by several possible methodological challenges. These difficulties are especially relevant in research on racial and ethnic disparities and neuropsychological outcomes because of the particular influence of selection and measurement in these contexts. In this article, we review the counterfactual framework for thinking about causal effects versus statistical associations. We emphasize that causal inferences are key to predicting the likely consequences of possible interventions, for example in clinical settings. We summarize a number of common biases that can obscure causal relationships, including confounding, measurement ceilings/floors, baseline adjustment bias, practice or retest effects, differential measurement error, conditioning on common effects in direct and indirect effects decompositions, and differential survival. For each, we describe how to recognize when such biases may be relevant and some possible analytic or design approaches to remediating these biases.     

Further evidence on the relationship between goal setting and expectancy theories

This investigation sought to clarify the relationship between goal setting and expectancy theories. Toward this end, (a) previous theoretical and empirical efforts at integrating these two theories are reviewed, (b) issues of causal ordering, measurement, and the meanings of those measures are examined, and (c) the results of an empirical study using a sample of 344 college students and an academic task are presented. Results illustrate how different operationalizations of expectancy and attractiveness ratings result in the measurement of different constructs and influence the significance and direction of empirical findings. For almost all operationalizations, however, significant relationships were evident between expectancy theory constructs and goal choice, goal commitment, and performance. Causal analyses using cross-lag correlations were inconclusive, but mediation and path analyses provided some evidence that goals mediate the effects of expectancy theory constructs on performance.     

Thinking Ahead: Assuming Linear Versus Nonlinear Personality-Criterion Relationships in Personnel Selection

Recent studies suggest that the form of some personality–performance relationships may be curvilinear, meaning that traditional top-down selection is inefficient in capitalizing on underlying personality–performance relationships. This study examines how mean performance is affected by how well the selection method is aligned with the nature of personality–criterion relationships. A simulation manipulated the linearity or nonlinear inflection point of predictor–criterion relationships, and several selection approaches were implemented that varied in level of congruence with these relationships. Results indicate that incongruence can produce notable decrements in mean performance under some conditions. Some evidence also suggests that decrements can be greater when linearity is assumed but relationships are nonlinear (vs. when nonlinearity is assumed but relationships are linear), selection ratios are smaller, and a single predictor is used.     

Charismatic leadership,environmental dynamism,and performance

Most studies relating charismatic leadership to performance have limitations concerning selection of criterion measures and investigation of moderators. Therefore, this study examines relationships between charismatic leadership and multiple performance outcomes under different levels of environmental dynamism (i.e., level of environmental uncertainty, degree of technological change) and per type of Chief Executive Officer (firm owner versus managing director). Results revealed that charismatic leadership was positively related to common-source and multi-source perceptual performance outcomes (i.e., subordinates' positive work attitude) and to organization profitability, but unrelated to organization liquidity and solvency. The relationship between charismatic leadership and perceptual performance was stronger under conditions of environmental uncertainty than under conditions of environmental certainty. Furthermore, charismatic leadership was more strongly related to organization profitability for firm owners than for managing directors who do not own their firm. The results are discussed and several potentially fruitful avenues for future research on charismatic leadership and employee as well as organizational performance are presented.     

Unidimensional versus multidimensional approaches to the assessment of acculturation for Asian American populations

This study used generational status and the Suinn-Lew Asian Self-Identity Acculturation scale to examine unidimensional versus multidimensional approaches to the conceptualization and measurement of acculturation and their relationships to relevant cultural indicator variables, including measures of Individualism-Collectivism, Independent-Interdependent Self-Construal, Loss of Face, and Impression Management. Multivariate analyses of covariance and partial correlations were used to examine the relationship between the acculturation models and each set of cultural indicator variables while controlling for socioeconomic status. Given that acculturation differences are often cited as evidence for a culture effect between groups, the present findings of an uneven nature of these relationships as a function of the particular acculturation measurement strategy have important implications for research on Asian Americans.     

The influence of monetary incentives on goal choice,goal commitment,and task performance

This study was designed to investigate the mechanism by which monetary incentives influence goal choice, goal commitment, and task performance. It is hypothesized that the explanation for incentive effects on goals and performance is in their influence on mediating events. A model attempting to explain the process of goal choice, work motivation, and performance in terms of cognitions is presented. Subjects were recruited for 5 days of part-time employment in a simulated organization to perform a clerical data transfer task. The 130 subjects were assigned randomly to 7 experimental conditions differing in terms of the magnitude of incentive offered for various levels of performance. The quantity and quality of performance was recorded daily and research questionnaires were administered at different times during their employment. Results provide support for the model presented and therefore suggest that the process of goal choice and commitment is central to understanding how incentives influence goals, motivation, and performance. The model provides a useful basis for investigating the relationships between organizational context and employee cognitions and for integrating goal setting with expectancy theory. The findings and future research issues are discussed.     

Faking under a nonlinear relationship between personality assessment scores and job performance

This study investigated whether faking in personality assessments affects the prediction of job performance and consequences of selection decisions when a nonlinear relationship was assumed between personality scores and job performance. It was also examined whether a selection strategy suggested for nonlinear relationships reduces the adverse impact of faking on consequences of selection decisions. The findings from the item response theory‐based simulations indicated that the prediction of job performance was affected by faking, whereas selection accuracy was not substantially affected by faking. The improvement in selection accuracy due to the strategy suggested for a nonlinear relationship was not noticeable. However, the suggested strategy reduced adverse impacts of faking on the estimation of selected applicants' performance scores.     

Assessment Center Graduate Selection: Decision Processes,Validity, and Evaluation by Candidates

In this contribution, results are presented of research on the predictive validity of a procedure for the career-oriented selection of graduates in The Netherlands. After selection on account of applicant letter, the selection procedure consists of four selective steps: interview with a ‘recruitment officer’, mental test, interview with line manager, and an assessment center comprising two situational exercises. Performance indicators investigated are present salary, average annual changes in function, appraisal of performance, appraisal of management potential, and average annual salary progression. We report on the relationships between the predictors, between the criteria, and between predictors and criteria. The assessment center, together with other steps from the selection procedure, appear to be valid for a number of performance indicators. The correlation between the overall assessment center rating and present salary, corrected for restriction in range, equals 0.55. Stepwise regression indicates that elements from all procedure steps contribute to the prediction of the criterion of average annual salary progression. The disturbing influence of possible ‘crown prince’ -effects on these results, are discussed. Also, the decision process taking place during the end meeting of the assessors is investigated. Finally, the evaluation of the procedure, both by accepted and by rejected candidates, is discussed. These and other results are compared to what is generally known about assessment centers.     

Do Impression Management Scales in Personality Inventories Predict Managerial Job Performance Ratings?

Theorists have argued that impression management scales should predict job performance in jobs where interpersonal interactions are important. This research examined the correlation between impression management scale scores and overall job performance as well as its component dimensions for managerial jobs (one type of job requiring considerable interpersonal interaction). First, a meta‐analysis was conducted to determine the relationship between impression management scale scores and overall managerial performance. Results indicated that the correlation was .04 (k = 17, N = 20,069) for managers. Two additional studies were conducted to investigate the relationship between impression management and specific facets or dimensions of managerial performance. In one of these two studies, the relationships between impression management scores and supervisory ratings on 24 performance facets were investigated for 826 managers. Criterion‐related validities ranged from ‐.06 to .07. In the second study, correlations of an impression management scale with ratings on 22 dimensions of performance were examined using a sample of 257 executives. The validities ranged from .01 to .11. There appears to be little evidence that impression management scales predict job performance in at least one job category where interpersonal interactions are important (management). Potential relevance of these results for understanding social desirability in personality measurement for personnel selection is discussed.