Reconsidering the Equivalence of Multisource Performance Ratings: Evidence for the Importance and Meaning of Rater Factors

Purpose

The study specified an alternate model to examine the measurement invariance of multisource performance ratings (MSPRs) to systematically investigate the theoretical meaning of common method variance in the form of rater effects. As opposed to testing invariance based on a multigroup design with raters aggregated within sources, this study specified both performance dimension and idiosyncratic rater factors.

Design/Methodology/Approach

Data was obtained from 5,278 managers from a wide range of organizations and hierarchical levels, who were rated on the BENCHMARKS^® MSPR instrument.

Findings

Our results diverged from prior research such that MSPRs were found to lack invariance for raters from different levels. However, same level raters provided equivalent ratings in terms of both the performance dimension loadings and rater factor loadings.

Implications

The results illustrate the importance of modeling rater factors when investigating invariance and suggest that rater factors reflect substantively meaningful variance, not bias.

Originality/Value

The current study applies an alternative model to examine invariance of MSPRs that allowed us to answer three questions that would not be possible with more traditional multigroup designs. First, the model allowed us to examine the impact of paramaterizing idiosyncratic rater factors on inferences of cross-rater invariance. Next, including multiple raters from each organizational level in the MSPR model allowed us to tease apart the degree of invariance in raters from the same source, relative to raters from different sources. Finally, our study allowed for inferences with respect to the invariance of idiosyncratic rater factors.     

The Performance Appraisal Milieu: A Multilevel Analysis of Context Effects in Performance Ratings

Purpose

The purpose of this study was to take an inductive approach in examining the extent to which organizational contexts represent significant sources of variance in supervisor performance ratings, and to explore various factors that may explain contextual rating variability.

Design/Methodology/Approach

Using archival field performance rating data from a large state law enforcement organization, we used a multilevel modeling approach to partition the variance in ratings due to ratees, raters, as well as rating contexts.

Findings

Results suggest that much of what may often be interpreted as idiosyncratic rater variance, may actually reflect systematic rating variability across contexts. In addition, performance-related and non-performance factors including contextual rating tendencies accounted for significant rating variability.

Implications

Supervisor ratings represent the most common approach for measuring job performance, and understanding the nature and sources of rating variability is important for research and practice. Given the many uses of performance rating data, our findings suggest that continuing to identify contextual sources of variability is particularly important for addressing criterion problems, and improving ratings as a form of performance measurement.

Originality/Value

Numerous performance appraisal models suggest the importance of context; however, previous research had not partitioned the variance in supervisor ratings due to omnibus context effects in organizational settings. The use of a multilevel modeling approach allowed the examination of contextual influences, while controlling for ratee and rater characteristics.

     

Testing Differential Mediation Effects of Sub-dimensions of Political Skills in Linking Proactive Personality to Employee Performance

Purpose  

The purpose of this research was to examine the mediating roles of political skill dimensions (i.e., networking ability, interpersonal influence, social astuteness, and apparent sincerity) in linking employees’ proactive personality and supervisor-rated in-role performance and altruism.     

Boredom Proneness: Its Relationship with Subjective Underemployment,Perceived Organizational Support,and Job Performance

Purpose  

The current study examined the relationship between trait boredom (i.e., boredom proneness), subjective underemployment, perceived organizational support, and job performance.     

Too Old to Train or Reprimand: The Role of Intergroup Attribution Bias in Evaluating Older Workers

Purpose

The purpose of this study was to explore the role rater and target age play in the evaluations of poorly performing workers. Intergroup attribution theory suggests that rater age predicts the attributions made for the poor performance of older workers.

Design/Methodology/Approach

In this study, 203 supervisors in various industries completed measures of causal attribution and evaluations for a poorly performing hypothetical subordinate.

Findings

Compared to the poor performance of younger targets, the poor performance of older targets was more likely to be attributed to external and controllable causes by older raters and more likely to be attributed to stable causes by younger raters. These attributions predicted willingness to punish and likelihood to provide training.

Implications

Our findings were partially supported by intergroup attribution theory and suggest that this theory may be useful in understanding how older workers’ performance is evaluated.

Originality/Value

This is one of the first studies to utilize intergroup attribution theory among supervisors in exploring how older workers are evaluated in the workplace and to demonstrate that the theory predicts how older workers’ poor performance will be attributed. Our study provides evidence that when evaluating a poorly performing older target, older raters will be more inclined to attribute this performance to controllable causes and thus be more punitive than younger raters. Further, we provide some evidence that raters will be more punitive and less willing to provide training when evaluating poorly performing targets to whom they are similar.     

A General Structure of Job Performance: Evidence from Two Studies

Purpose  

The purpose of this study is to test alternative models of job performance based on competing categorization criteria.     

Factors Influencing Employee Intentions to Provide Honest Upward Feedback Ratings

Based on current research involving rater motivation, we examined several factors hypothesized to influence employee intentions to provide honest upward feedback. Survey data were collected from a demographically diverse sample of hospital employees (n = 203). In summary, we found empirical support for generalizing extant models of rater motivation to an upward feedback context: cynicism towards upper management and the upward feedback process, understanding upward feedback, and opportunity to observe their supervisors were the primary predictors of employee intentions to provide honest upward feedback ratings, mediated by the (a) extent to which employees perceived positive benefits would result from rating their supervisors honestly, (b) the extent to which employees feared retaliation by their supervisors, and (c) rater self-efficacy.

Effects of Retail Store Image Attractiveness and Self-Evaluated Job Performance on Employee Retention

Purpose  

The purpose of this research was to investigate the effects of store image attractiveness and self-evaluated job performance on internal job satisfaction and organizational commitment.     

Effects of Applicant Personality on Resume Evaluations

Purpose

This study examined the extent that personality information in resumes impacts hiring judgments through applicant’s resumes. Study 1 examined lay theories regarding relationships between resume cues and the applicant’s personality and hireability. Study 2 examined how the applicant’s personality impacted hiring judgments through resumes.

Design/Methodology/Approach

Data for both studies were collected in the context of a managerial position. For Study 1, participants assessed resume cues in regards to their relationship with personality and hireability. For Study 2, Human Resource personnel evaluated each resume in regards to personality and hireability.

Findings

Results for Study 1 highlight several connections between applicants’ personality and resumes, with strong links between resume content and perceptions of conscientiousness and agreeableness. Results for Study 2 indicate that personality was largely unrelated to ratings of hireability but perceptions of personality were strongly linked to hireability; actual personality was linked to the variability in cue information related to hireability, and conscientiousness was indirectly related to hireability through judgments of conscientiousness.

Implications

Results from these studies suggest that personality and perceptions of personality play a greater role in resume development and screening than has been previously suggested. The pattern of results reported suggest that there are a number of resumes cues that accurately reflect an applicant’s personality and influence perceptions of hireability.

Originality/Value

By taking an exploratory approach, the current studies were able to explore a large variety of cues linked to personality and ratings of hireability. Results have implications for both applicants and HR personnel evaluating resumes.     

A Dispositional Approach to Work�CSchool Conflict and Enrichment

Purpose

The purpose of this study was to investigate a dispositional model of the work?Cschool interface. In particular, we examined the relationship between core self-evaluations (CSEs) and proactive personality with both work?Cschool conflict (WSC) and work?Cschool enrichment (WSE) as well as a variety of work and school outcomes.

Design/methodology/approach

314 employed college students were surveyed about their experiences managing work and school roles.

Findings

Structural equation modeling showed that CSEs were related to both WSC and WSE as well as school outcomes, whereas proactive personality was related to WSE and job and school outcomes but not WSC. WSC was negatively related to school performance, whereas WSE was positively related to school and job satisfaction as well as school performance.

Implications

Organizations, universities, and researchers should be aware that dispositional variables influence perceptions of work and school roles as well as important outcome variables, and that further intervention efforts may be needed to help students manage work and school roles.

Originality/Value

Past studies have demonstrated that job characteristics influence the work?Cschool interface, but this study is among the first to demonstrate that dispositional factors also relate to WSC and WSE.     

Examining the Relationships Between Personality,Coping Strategies,and Work–Family Conflict

Purpose  

The purpose of this study was to examine the processes through which personality characteristics may influence work–family conflict (WFC). Specifically, the mediating effects of selection, optimization, and compensation (SOC) behavioral stress-coping strategies on the relationship between personality characteristics and WFC were tested.     

The Romance of Learning from Disagreement. The Effect of Cohesiveness and Disagreement on Knowledge Sharing Behavior and Individual Performance Within Teams

Purpose  

The purpose of this study was to explore the effects of disagreement and cohesiveness on knowledge sharing in teams, and on the performance of individual team members.     

Psychological Contracts and Counterproductive Work Behaviors: Employee Responses to Transactional and Relational Breach

Purpose  

This study extends the research on counterproductive work behavior (CWB) by examining the psychological contract breaches that trigger employee CWB. Specifically, we explored the relationship between transactional and relational contract breach and five forms of CWB (abuse, production deviance, sabotage, theft, and withdrawal). Further, we considered the role of situational and individual factors that mitigate CWB engagement and examined the moderating effects of organizational policies meant to deter CWB and personality (conscientiousness, agreeableness, and emotional stability).     

Differential Effect of Inter-Role Conflict on Proactive Individual’s Experience of Burnout

Purpose

This study examined how proactive personality interacts with inter-role conflict, measured as work?Cfamily conflict and family?Cwork conflict, to predict burnout, measured as emotional exhaustion, depersonalization, and reduced personal accomplishment.

Design/Methodology/Approach

Participants were 171 clerical employees. Hierarchical multiple regression was used to test whether proactive personality moderated the relationship between inter-role conflict and forms of burnout.

Findings

Family?Cwork conflict was not associated with burnout, but work?Cfamily conflict explained 30% of unique variance in emotional exhaustion and 9% in depersonalization. Proactive personality explained 12% of variance in personal accomplishment. Three-way interactions indicated that at high levels of work??family conflict and family??work conflict, proactive individuals reported lower levels of emotional exhaustion and personal accomplishment and higher levels of depersonalization than less proactive individuals.

Implications

While previous research has generally documented the virtues of proactive personality, our research indicates that when simultaneously faced with work?Cfamily and family?Cwork conflict, individuals with proactive personality experience more depersonalization and less personal accomplishment relative to less proactive individuals. Overall, results of three-way interactions imply that while a certain level of proactive personality may be necessary to buffer feelings of emotional exhaustion, beyond a certain level, proactive personality may lead one to experience higher levels of depersonalization and reduced personal accomplishment.

Originality/Value

This study extends previous research by examining the influence of two types of inter-role conflict on all three dimensions of burnout. It also responds to calls for additional research on potential moderators, buffers, or even antidotes to stress by examining how proactive personality interacts with stressors.     

The Moderating Effect of Personality on Employees’ Reactions to Procedural Fairness and Outcome Favorability

Purpose  

The purpose of this study was to examine how personality moderates the interactive effect of procedural fairness perceptions and outcome favorability on employees’ job attitudes.     

Reflection Toward Creativity: Team Reflexivity as a Linking Mechanism Between Team Goal Orientation and Team Creative Performance

Purpose

This study aims at testing the mediating role of team reflexivity in the relationships between team learning, performance-prove, and performance-avoid goal orientations and team creative performance and assessing the relative importance of the three types of team goal orientation in team reflexivity and creative performance.

Methodology

We conducted Study 1 on 68 student teams by using a two-wave time-lagged design. In Study 2, we carried out a cross-sectional field study on 108 intact work teams in diverse Korean companies.

Findings

Team learning goal orientation was significantly associated with team creative performance. While team learning and performance-prove goal orientations were equally influential in predicting team reflexivity, team performance-avoid goal orientation had no relationship with team reflexivity and creative performance. Team reflexivity mediated the relationships between team learning and performance-prove goal orientations and team creative performance.

Implications

By revealing that team learning and performance-prove goal orientations can contribute to team creative performance through the facilitation of team reflective process, this study provides practitioners with insight into critical antecedents and team process that are conducive to the creative performance of work teams.

Originality/Value

This is one of the first studies to explore a mediating mechanism between team goal orientation and creative performance. This study attends to the role of team reflexivity as a key team-regulatory process that underlies the relationship between team goal orientation and team performance. Furthermore, the use of multiple studies in different contexts strengthens the robustness of the study findings.

     

A Construct Validation of the Neutral Objects Satisfaction Questionnaire (NOSQ)

Purpose  

We address concerns regarding the affective-orientation of affective disposition; a personality characteristic assessed using the Neutral Objects Satisfaction Questionnaire (NOSQ). In addition, we examine the utility of affective disposition while controlling for other affective-oriented personality characteristics.     

Differential Effects of Task Performance,Organizational Citizenship Behavior,and Job Complexity on Voluntary Turnover

Purpose

The purpose of this study was to investigate the performance–turnover relationship by considering the effects of task performance and OCBs simultaneously while also examining the moderating effect job complexity has on the relationship between voluntary turnover and each type of performance.

Design/Methodology/Approach

Data were obtained as part of a larger study to validate an employment test, in which actual turnover data and supervisory ratings of job performance were collected for employees in two hospitals (n = 782).

Findings

Task performance exhibited a curvilinear relationship with turnover, while OCB exhibited a negative linear relationship with turnover. Job complexity moderated both of these relationships. For task performance, turnover in high-complexity jobs was greater for low performers but lower for high performers relative to that of employees in low-complexity jobs. For OCB, the negative relationship with turnover was more pronounced in high-complexity jobs.

Implications

Both low- and high-task performers are more likely to turnover, while employees exhibiting high OCBs are less likely to turnover. These results imply that retention strategies are critical for top performers, but especially in high-complexity jobs. Organizations may be able to discourage voluntary turnover by creating conditions that stimulate OCB, particularly in highly complex jobs.

Originality/Value

Most prior performance–turnover relationship research used unidimensional measures of performance, whereas this study included two dimensions of performance and examined this relationship while controlling for one-performance dimension when predicting the other. Furthermore, this study is one of the first studies to suggest that job complexity moderates the performance–turnover relationship.

     

The Impact of Applicant Faking on Selection Measures,Hiring Decisions,and Employee Performance

Purpose

The purpose of this study was to examine the prevalence of applicant faking and its impact on the psychometric properties of the selection measure, the quality of hiring decisions, and employee performance.

Design/Methodology/Approach

This study utilized a within-subjects design where responses on a self-report measure were obtained for 162 individuals both when they applied for a pharmaceutical sales position, and after they were hired. Training performance data was collected at the completion of sales training and sales data was collected 5 months later.

Findings

Applicant faking was a common occurrence, with approximately half of the individuals being classified as a faker on at least one of the dimensions contained in the self-report measure. In addition, faking was found to negatively impact the psychometric properties of the selection measure, as well as the quality of potential hiring decisions made by the organization. Further, fakers exhibited lower levels of performance than non-fakers.

Implications

These findings indicate that past conclusions that applicant faking is either uncommon or does not negatively impact the selection system and/or organizational performance may be unwarranted.

Originality/Value

Remarkably few studies have examined applicant faking using a within-subjects design using actual job applicants, which has limited our understanding of applicant faking. Even fewer studies have attempted to link faking to criterion data to evaluate the impact of faking on employee performance. By utilizing this design and setting, the present study provides a unique glimpse into both the prevalence of faking and the significant impact faking can have on organizations.     

Situating Ethical Behavior in the Nomological Network of Job Performance

Purpose

Our objective was to generate, define, and evaluate behavioral dimensions of ethical performance at work that are common across United States occupations.

Design/Methodology/Approach

This project involved three studies. Study 1 involved (a) qualitative review of published literature, professional codes of ethics, and critical incidents of (un)ethical performance and resulted in (b) behavioral dimensions and ethical performance rating scales. The second and third studies used a retranslation methodology to evaluate the ethical performance dimensions from Study 1. The behavioral dimensions were linked to the performance determinants (personal attributes) in Study 3.

Findings

Study 1 resulted in draft dimension definitions and rating scales for 10 ethical performance dimensions. In Studies 2 and 3, retranslation data provided strong support for 10 behavioral dimensions of ethical performance at work. Results from Study 3 shed light on possible relationships among the performance dimensions based on their underlying performance determinants.

Implications

Communicating an organization’s ethical standards to employees is important because some ethical breakdowns can be attributed to simply failing to recognize an ethical matter (in: DeCremer, Managerial ethics: Managing the psychology of morality, Routledge, New York, 2011). Definitions of ethical behavior in the workplace provide a tool for researchers, employers, and employees to communicate about ethical situations and a foundation for folding ethics into employee training and performance management.

Originality/Value

These studies provide a taxonomy of ethical performance at work that generalizes to a diverse array of occupations and industries, and dimensions and rating scales have value for performance management, training/curriculum development, job analysis, predictor development and/or validation, and additional research.