360 Degree Feedback and Developmental Outcomes: The Role of Feedback Characteristics,Self‐Efficacy and Importance of Feedback Dimensions to Focal Managers' Current Role

This longitudinal study investigates whether developmental changes following 360 degree feedback are predicted by the favourability of ratings received, and moderated by focal individuals' self‐efficacy and perceived importance of feedback. Five developmental criteria are investigated longitudinally: (i) self‐assessments, (ii) line managers' ratings, (iii) amount of developmental activity, (iv) global self‐efficacy and (iv) self‐efficacy for development. Feedback ratings from certain rater groups predicted changes in ratings, but not changes in self‐efficacy or amount of developmental activity. Self‐efficacy significantly moderated the feedback–performance association for certain rater groups, but feedback importance did not. Contrary to expectations, the focal individual's initial self‐assessment predicted changes in self‐efficacy, over the favourability of ratings received. The implications of these findings for organizations using 360 degree feedback for developmental purposes are discussed.     

Self‐Supervisor Agreement: The Influence of Feedback Seeking on the Relationship Between Self and Supervisor Ratings of Performance1

Self‐assessment research has continued to search for those factors that increase self‐other rating agreement. The current field study investigated the feedback‐seeking strategies (i. e., monitoring and inquiry) used by 125 employees to obtain performance information, as well as the relationship between feedback‐seeking strategy use and self‐supervisor performance‐rating agreement. Results indicate that the frequency of monitoring reported by employees significantly moderated the relationship between self and supervisor ratings of performance. Individuals who reported higher levels of feedback seeking through monitoring were more likely to have self‐assessments that were congruent with their supervisors' ratings of performance.     

Same‐gender and same‐race bias in assessment center ratings: A rating error approach to understanding subgroup differences

This study investigated leniency and similar‐to‐me bias as mechanisms underlying demographic subgroup differences among assessees in assessors’ initial dimension ratings from three assessment center (AC) simulation exercises used as part of high‐stakes promotional testing. It examined whether even small individual‐level effects can accumulate (i.e., “trickle‐up”) to produce larger subgroup‐level differences. Individual‐level analyses were conducted using cross‐classified multilevel modeling and conducted separately for each exercise. Results demonstrated weak evidence of leniency toward White assessees and similar‐to‐me bias among non‐White assessee–assessor pairs. Similar leniency was found toward female assessees, but no statistically significant effects were found for assessee or assessor gender or assessee–assessor gender similarity. Using traditional d effect size estimates, weak individual level assessee effects translated into small but consistent subgroup differences favoring White and female assessees. Generally small but less consistent subgroup differences indicated that non‐White and male assessors gave higher ratings. Moreover, analyses of overall promotion decisions indicate the absence of adverse impact. Findings from this AC provide some support for the “trickle‐up” effect, but the effect on subgroup differences is trivial. The results counter recent reviews of AC studies suggesting larger than previously assumed subgroup differences. Consequently, the findings demonstrate the importance of following established best practices when developing and implementing the AC method for selection purposes to minimize subgroup differences.     

Common mechanisms of executive attention underlie executive function and effortful control in children

Executive Function (EF) and Effortful Control (EC) have traditionally been viewed as distinct constructs related to cognition and temperament during development. More recently, EF and EC have been implicated in top‐down self‐regulation ‐ the goal‐directed control of cognition, emotion, and behavior. We propose that executive attention, a limited‐capacity attentional resource subserving goal‐directed cognition and behavior, is the common cognitive mechanism underlying the self‐regulatory capacities captured by EF and EC. We addressed three related questions: (a) Do behavioral ratings of EF and EC represent the same self‐regulation construct? (b) Is this self‐regulation construct explained by a common executive attention factor as measured by performance on cognitive tasks? and (c) Does the executive attention factor explain additional variance in attention deficit hyperactivity disorder (ADHD) problems to behavioral ratings of self‐regulation? Measures of performance on complex span, general intelligence, and response inhibition tasks were obtained from 136 preadolescent children (M = 11 years, 10 months, SD = 8 months), along with self‐ and parent‐reported EC, and parent‐reported EF, and ADHD problems. Results from structural equation modeling demonstrated that behavioral ratings of EF and EC measured the same self‐regulation construct. Cognitive tasks measured a common executive attention factor that significantly explained 30% of the variance in behavioral ratings of self‐regulation. Executive attention failed to significantly explain additional variance in ADHD problems beyond that explained by behavioral ratings of self‐regulation. These findings raise questions about the utility of task‐based cognitive measures in research and clinical assessment of self‐regulation and psychopathology in developmental samples.     

Assessor Constructs in Use as the Missing Component in Validation of Assessment Center Dimensions: A critique and directions for research

The basis for assessment center dimension ratings has been examined through the lens of various multitrait–multimethod approaches, with some researchers concluding that dimension ratings are not representative of meaningful constructs. This presents a serious challenge for those who would generalize predictor constructs, particularly those that are broadly articulated in management assessment center dimensions. This paper addresses the problems with applying such analyses to assessment center dimension ratings without first articulating expected construct relationships. Ackerman and Humphreys' analysis of constructs will be used to better frame assessment center dimensions and to articulate possible assessor constructs in use. Theories of performance held by the managers, who serve both as subject matter experts when dimensional constructs are initially defined, and as the assessors who impose these dimensions on assessees' behavior in assessment center ratings, are suggested as important bases for further construct validation. Approaches for generalization of assessor constructs in use to the managerial work domain will be discussed.     

SELF- VERSUS OTHERS' RATINGS AS PREDICTORS OF ASSESSMENT CENTER RATINGS: VALIDATION EVIDENCE FOR 360-DEGREE FEEDBACK PROGRAMS

Although 360-degree feedback programs are rapidly increasing in popularity, few studies have examined how well ratings from these programs predict an independent criterion. This study had 2 main aims: First, to examine the validity of ratings from a 360-degree feedback program using assessment center ratings as an independent criterion and to determine which source (i.e., self, supervisor, peers, or subordinates) provided the most valid predictor of the criterion measure of competency. Second, to better understand the relationship between self-observer discrepancies and an independent criterion. The average of supervisor, peer, and subordinate ratings predicted performance on the assessment center, as did the supervisor ratings alone. The self-ratings were negatively and nonlinearly related to performance with some of those who gave themselves the highest ratings having the lowest performance on the assessment center. Supervisor ratings successfully discriminated between overestimators but were not as successful at discriminating underestimators, suggesting that more modest feedback recipients might be underrated by their supervisors. Peers overestimated performance for poor performers. Explanations of the results and the implications for the use of self-ratings in evaluations, the design of feedback reports, and the use of 360-degree feedback programs for involving and empowering staff are discussed.     

A Multi‐Source,Multi‐Study Investigation of Job Performance Prediction by Political Skill

Political skill is a social effectiveness construct with a demonstrated capacity to predict job performance. However, because performance prediction research in this area to date has made exclusive use of self‐reports of political skill, and due to frequent distrust of self‐ratings of constructs in important personnel decisions, there is a need to investigate how multiple alternative sources of political skill and job performance measures relate, thus raising both theoretical and methodological issues. In three studies, employing a triadic data collection methodology, and utilising both cross‐sectional and longitudinal designs, this research tested the hypotheses that employee political skill, measured from the perspective of employees' assessor A, will positively predict job performance rated by assessor B (i.e. Hypothesis 1a), and vice versa, that employee political skill measured by assessor B will predict job performance ratings measured by assessor A (i.e. Hypothesis 1b).     

Self‐ratings in Training Programs: An Examination of Level of Performance and the Effects of Feedback

The level of agreement (mean differences and correlations) between self, peer and training staff ratings were examined in this study. The sample consisted of 545 participants who were undertaking a Royal Australian Airforce officer training program. Consistent with previous research there was strong agreement between training staff and peers and weak agreement between self‐ratings and ratings by others (training staff and peers). Accuracy of ratings was examined by (a) comparing the mean ratings of outstanding, average and below‐average performers; and (b) correlating difference scores with a measure of performance. The findings showed that below‐average performers have a less accurate view of themselves compared to outstanding performers. Finally, we examined the effects of negative feedback on self‐perceptions. The analyses indicated that after receiving negative feedback, average performers adjusted their self‐ratings. Various explanations were proposed together with practical implications for training.     

Parenting and child behaviour problems: a longitudinal analysis of non‐shared environment

This study examined potential non‐shared environmental processes in middle childhood by estimating statistical associations between monozygotic (MZ) twin differences in externalizing and internalizing problems and positive social engagement, and differential maternal positivity and negativity, over 1 year. Seventy‐seven pairs of identical twins participated (M=6.08‐years old, 65% male) in two annual home visits. Observers' ratings and maternal reports were gathered. At both assessments, the twin who showed more conduct problems (maternal report and observers' ratings) and less positive social engagement (positive affect, responsiveness) received more maternal negativity and less maternal warmth (self‐reports and observers' ratings), relative to his or her genetically identical co‐twin. The same patterns held over time, for the associations between change in differential MZ twin conduct problems and social engagement and change in differential maternal behaviour. Effects for child internalizing problems were not consistent within or across raters. Overall, these results indicated that differential maternal warmth and negativity—self‐perceived and observed by others—are important aspects of sibling differentiation for both problematic and adaptive behaviours during middle childhood. Copyright © 2009 John Wiley & Sons, Ltd.     

Students Aggress Against Professors in Reaction to Receiving Poor Grades: An Effect Moderated by Student Narcissism and Self‐Esteem

Laboratory evidence about whether students’ evaluations of teaching (SETs) are valid is lacking. Results from three (3) independent studies strongly confirm that "professors" who were generous with their grades were rewarded for their favor with higher SETs, while professors who were frugal were punished with lower SETs (Study 1, d = 1.51; Study 2, d = 1.59; Study 3, partial η² = .26). This result was found even when the feedback was manipulated to be more or less insulting (Study 3). Consistent with laboratory findings on direct aggression, results also indicated that, when participants were given a poorer feedback, higher self‐esteem (Study 1 and Study 2) and higher narcissism (Study 1) were associated with them giving lower (more aggressive) evaluations of the "professor." Moreover, consistent with findings on self‐serving biases, participants higher in self‐esteem who were in the positive grade/feedback condition exhibited a self‐enhancing bias by giving their "professor" higher evaluations (Study 1 and Study 2). The aforementioned relationships were not moderated by the professor's sex or rank (teaching assistant vs.professor). Results provide evidence that (1) students do aggress against professors through poor teaching evaluations, (2) threatened egotism among individuals with high self‐esteem is associated with more aggression, especially when coupled with high narcissism, and (3) self‐enhancing biases are robust among those with high self‐esteem. Aggr. Behav. 39:71‐84, 2013. © 2012 Wiley Periodicals, Inc.     

Multisource Assessments of Behavioral Competencies and Selection Interview Performance

This study sought to determine whether and how ratings of leadership in a developmental multisource feedback program compare against those obtained by leadership selection decision makers. Seventy‐seven senior managers received feedback on eight competencies that also formed the basis of a subsequent promotional interview for senior executive positions. Each rating source displayed a unique perspective on the participants, but only the supervisor and peer ratings predicted performance on the interview. As well, peer, subordinate, and self‐ratings failed to improve upon the supervisors' predictions. The results also suggested a general performance factor underlying both the multisource assessments and the interview ratings.     

How leaders stimulate employee learning: A leader–member exchange approach

This study investigated how leader–member exchange (LMX), goal setting, and feedback are related to employee engagement in learning activities. Two different mechanisms were proposed: a mediating mechanism holding that LMX elicits specific leader behaviours (i.e., goal setting and feedback) which would mediate the LMX‐learning relationship, and a moderating mechanism, holding that LMX would strengthen the effect of these leader behaviours. A sample of 1,112 employees from 7 organizations completed questionnaires that measured LMX, goal specificity, feedback, and self‐reports of employee engagement in learning activities. The 233 direct leaders of these employees completed questionnaires that measured goal difficulty and leader ratings of employee engagement in learning activities. Multi‐level analysis showed that goal difficulty and goal specificity mediated the relationship between LMX and employee engagement in learning activities, and that LMX moderated the relationship of goal difficulty with employee engagement in learning activities. With these findings, the present study contributes to the literatures on LMX, goal setting, and employee development.     

Analyzing Judgments of Ethnically Diverse Applicants During Personnel Selection: A study at the Dutch police

A judgment‐analysis study was used to investigate assessors' judgment processes, evaluating ethnic minority vs ethnic majority applicants. Sixteen ethnic majority assessors judged 5089 applicants during the Dutch police officer selection procedure, with each assessor judging 30 ethnic minority applicants minimally. Information from an employment interview, an assessment center, and a Big Five personality test were combined into a final selection advice. Results showed that as much as or more information sources were used to judge ethnic minority than ethnic majority applicants. Furthermore, a larger number of irrelevant cues were used for the judgment of ethnic minority applicants. Finally, when judging ethnic minority applicants, assessors based their decision to a lesser extent on their own ratings than on ratings of others.     

Linking executive function skills and physiological challenge response: Piecewise growth curve modeling

This study employed piecewise growth curve modeling to examine how children's executive function (EF) skills relate to different components of children's physiological response trajectory – initial arousal, reactivity, and recovery. The sample included 102 ethnically diverse kindergarteners, whose EF skills were measured using standard tasks and observer ratings. Physiological response was measured via changes in respiratory sinus arrhythmia (RSA) in response to a laboratory socio‐cognitive challenge. Children's cool and hot EF skills were differentially related to both linear and quadratic components of RSA response during the challenge. Greater hot EF skills and assessor report of EF skills during laboratory visit were related to quicker RSA recovery after the challenge. These findings demonstrate that children's physiological response is a dynamic process that encompasses physiological recovery and relates to children's self‐regulation abilities.     

The Effectiveness of Collaborative/Therapeutic Assessment for Psychotherapy Consultation: A Pragmatic Replicated Single-Case Study

This pragmatic study evaluated the effectiveness of a collaborative assessment intervention as an approach to midtherapy consultation, which has yet to be empirically tested. Ten adult participants in ongoing psychotherapy with a variety of presenting concerns, primarily consisting of general mood and adjustment issues, received a brief assessment-based intervention based on Finn's (2007) Therapeutic Assessment model. Following the collection of assessment questions and the administration of a multimethod assessment battery, clients and therapists participated in a joint feedback session with the assessor. Clients were then followed as their psychotherapy continued. The results of idiographic and aggregate analytic approaches revealed significant reductions in client-reported symptomatic distress, as evidenced by a medium effect size (d = –.50) and a significant change in the trajectory of distress. Client reports of the process of psychotherapy revealed a significant increase in the clients’ ratings of the working alliance. The findings suggest that a midtherapy consultation using collaborative/therapeutic assessment methods is beneficial but that further rigorous investigation is needed.     

TRAINING COUNSELING SKILLS: AN EXPERIMENTAL ANALYSIS AND SOCIAL VALIDATION

A community development perspective suggests the value of using local resources to solve local problems. Two low-income staff of a community service center served as nonprofessional counselors after receiving a training program consisting of written instructions, practice, and performance feedback. The effects of the 40-hour training program were positive for both counseling and problem-solving skills. There was also evidence of generalization of counseling performance across clients, problems, and time. In addition, expert judges' ratings of performance were obtained as a measure of social validity. These findings suggest that the training procedures are effective in enhancing the counseling and problem-solving skills of low-income nonprofessionals.     

The Positive Feedback Bias as a Response to Out‐Group Unfriendliness1

This study tests whether Whites provide especially positive feedback to minorities who respond to feedback in an unfriendly manner. White female undergraduates (N = 66) gave verbal feedback to either a Black or a White confederate who posed as the author of a poorly written essay. Confederates responded to participants’ feedback in either a friendly or unfriendly manner. As predicted, participants who gave feedback to an unfriendly Black confederate supplied a selectively higher ratio of positive to negative comments and a selectively higher proportion of positive comments. Participants paired with an unfriendly Black confederate also provided the most positive post‐interaction ratings, despite minimal impression‐management pressures. Collectively, these findings indicate that Whites’ self‐image motives underlie the positive feedback bias.     

Effects of personality and gender on self–other agreement in ratings of leadership

We explore the role of leader personality (i.e., the Big 5 traits: Conscientiousness, Agreeableness, Openness, Extraversion, and Neuroticism) and gender in self–other (dis)agreement (SOA) in ratings of leadership. We contend that certain aspects of the leader's persona may be more or less related to self‐ or other‐ratings of the leader's behaviour if those aspects are (1) more or less observable by others, (2) more or less related to internal thoughts versus external behaviours, (3) more or less prone to self‐enhancement or self‐denigrating biases, or (4) more or less socially desirable. We utilize statistical methodologies that capture fully the effects of multiple independent variables on the congruence between two dependent variables (Edwards, 1995 , Organizational Behavior and Human Decision Processes, 64, 307), which previously have not been applied to this area of research. Our results support hypotheses predicting less SOA as leader Conscientiousness increases and greater SOA as Agreeableness and Neuroticism increase. Additionally, we found gender to be an important factor in SOA; female leaders exhibited greater SOA than did their male counterparts. We discuss the implications of these findings, limitations, and future research directions.

Practitioner points

• Popular practices such as 360‐degree feedback may reveal discrepancies between a person's self‐ratings and other's ratings.
• Although often attributed to a lack of self‐awareness, these discrepancies also may be explained by factors such as the personality and gender of the focal individual.

     

Self–Observer Rating Discrepancies of Managers in Asia: A study of derailment characteristics and behaviors in Southern and Confucian Asia

Antecedents to self–observer rating discrepancies in multisource instruments have been established at the individual and organizational level. However, research examining cultural antecedents is limited, which is particularly relevant as multisource instruments gain popularity around the world. We investigated multisource ratings of 860 Asian managers from the regions of Southern Asia (n=261) and Confucian Asia (n=599) and analyzed cultural differences in self–observer rating discrepancies. Multivariate regression procedures revealed that the self–observer rating discrepancy was wider for managers from Southern Asia as compared with Confucian Asia. The reason for the discrepancy was driven by managers' self‐ratings being different across cultures than by observer ratings from managers' bosses, direct reports, or peers; the predictor is related to self‐ratings not observer ratings, producing differential self–observer ratings due to self‐ratings. We discuss cultural differences in self‐ and observer ratings within Asia and provide implications for the practice of multisource assessments.