Self–Observer Rating Discrepancies of Managers in Asia: A study of derailment characteristics and behaviors in Southern and Confucian Asia

Antecedents to self–observer rating discrepancies in multisource instruments have been established at the individual and organizational level. However, research examining cultural antecedents is limited, which is particularly relevant as multisource instruments gain popularity around the world. We investigated multisource ratings of 860 Asian managers from the regions of Southern Asia (n=261) and Confucian Asia (n=599) and analyzed cultural differences in self–observer rating discrepancies. Multivariate regression procedures revealed that the self–observer rating discrepancy was wider for managers from Southern Asia as compared with Confucian Asia. The reason for the discrepancy was driven by managers' self‐ratings being different across cultures than by observer ratings from managers' bosses, direct reports, or peers; the predictor is related to self‐ratings not observer ratings, producing differential self–observer ratings due to self‐ratings. We discuss cultural differences in self‐ and observer ratings within Asia and provide implications for the practice of multisource assessments.     

On Being the Same in Different Places: Evaluating Frame‐of‐Reference Effects Across Two Social Contexts

Frame‐of‐reference (FOR) effects in personality assessment are demonstrated when self‐rated items oriented to specific contexts (e.g., workplace) show better predictive validity than noncontextualized items. Empirical support of FOR effects typically relies on job performance ratings or academic grades for criteria. The current study evaluates FOR effects using ratings of personality provided by informants from the home or school context. Items from the NEO Five‐Factor Inventory (NEO‐FFI; Costa & McCrae, 1992) were contextualized to the home and school contexts to create NEO‐Home and NEO‐School versions. One hundred fifty‐eight college students completed the NEO‐Home and NEO‐School questionnaires, and 161 college students completed the standard, noncontextualized NEO‐FFI. All participants recruited one peer from college and at least one parent to complete standard rater versions of the NEO‐FFI. Contextualized self‐ratings did not show FOR effects. NEO‐Home self‐ratings did not correlate higher with parent ratings than with peer ratings, and NEO‐School self‐ratings did not correlate higher with peer ratings than with parent ratings. Standard NEO‐FFI self‐ratings generally showed higher self‐informant agreement with both types of informants than contextualized self‐ratings. The pattern of correlations suggests that validity is enhanced more by specific trait‐informant combinations than by the contextualization of items to social contexts.     

Informant reports add incrementally to the understanding of the perfectionism–depression connection: Evidence from a prospective longitudinal study

Perfectionism is a putative risk factor for depressive symptoms. However, most research in this area uses cross-sectional designs (which fail to address temporal precedence) and mono-source designs (which are influenced by various biases). The present study overcomes these limitations by using a novel design involving both self- and informant reports of self-critical perfectionism (i.e., negative reactions to perceived failures, concern over others’ criticism and expectations, doubts about performance abilities, and intense self-rebuke). It was hypothesized that self- and informant reports of self-critical perfectionism would correlate moderately and that self- and informant reports of self-critical perfectionism would predict increases in depressive symptoms over time. A sample of 155 target participants and 588 informants was recruited and studied using a prospective longitudinal design. All study hypotheses were supported, including evidence that self- and informant reports of self-critical perfectionism each add incrementally to the understanding of the self-critical perfectionism–depressive symptoms connection. Informant reports may provide a more complete picture of the self-critical perfectionist and her or his vulnerability to depressive symptoms.     

Measuring Context-Specific and Cross-Contextual Components of Hierarchical Constructs

Rating scales are often used to measure behavioral constructs. Yet, different informants’ ratings may not necessarily agree. The situational specificity (SS) perspective postulates that discrepancies between ratings by different informants are primarily attributable to contextual behavior of the people being rated. The multitrait-multimethod (MTMM) perspective, however, attributes discrepancies between informants to rater bias, i.e., each informant provides a systematically distorted picture of the person being rated. Similarly, the Attribution-Bias-Context (ABC) perspective also attributes informant discrepancies to systematic biases. Within the context of measuring hierarchical constructs, we proposed a hybrid perspective that takes account of variance attributable to the behavior of the person being rated in a particular context from the perspective of a specific informant. We then provided a parametric representation of this perspective and analyses of mother, teacher, and self-ratings of Rule-Breaking and Aggressive Behavior to illustrate features of the model. Strengths and limitations of the SS, MTMM, and hybrid perspectives are discussed.     

DEVELOPMENT ENGAGEMENT WITHIN AND FOLLOWING DEVELOPMENTAL ASSESSMENT CENTERS: CONSIDERING FEEDBACK FAVORABILITY AND SELF–ASSESSOR AGREEMENT

This study sought to understand employees' level of behavioral engagement in response to feedback received in developmental assessment center (DAC) programs. Hypotheses were drawn from theories of self‐enhancement and self‐consistency and from findings in the multisource feedback and assessment center literatures regarding recipients' perceptions of feedback. Data were gathered from 172 U.S. middle managers participating in a DAC program. Results suggested that more favorable feedback was related to higher behavioral engagement. When discrepancies between self‐ and assessor ratings were examined, overraters (participants whose overall self‐ratings were higher than their assessor ratings) tended to show less engagement in the program compared to underraters. However, pattern agreement on the participant's dimension profile did not significantly correlate with behavioral engagement. Based on these findings, avenues for future research are presented and practical implications are discussed.     

Multisource Assessments of Behavioral Competencies and Selection Interview Performance

This study sought to determine whether and how ratings of leadership in a developmental multisource feedback program compare against those obtained by leadership selection decision makers. Seventy‐seven senior managers received feedback on eight competencies that also formed the basis of a subsequent promotional interview for senior executive positions. Each rating source displayed a unique perspective on the participants, but only the supervisor and peer ratings predicted performance on the interview. As well, peer, subordinate, and self‐ratings failed to improve upon the supervisors' predictions. The results also suggested a general performance factor underlying both the multisource assessments and the interview ratings.     

The Impact of the Big Five Personality Traits on Reports of Child Behavior Problems by Different Informants

The present study compared ratings of a standardized sample of child behavior problems across informants and examined the effects of informant personality traits on child behavior ratings by mothers, teachers, and group-care workers. Participants were 55 clinic-referred children, aged 6–12 years. All informants watched and rated the same 17-min videotaped behavior sample of a familiar target child. Independent trained observers rated the same videotapes to provide criterion ratings. Informants personality traits were assessed using the NEO Five Factor Personality Inventory. Results showed that mothers reported fewer behavior problems than did the professionals, that the informants who were familiar with the child reported more behavior problems than did the independent observers, and that higher levels of informant neuroticism were related to higher ratings of child behavior problems in the case of the professionals, but not in the case of the mothers. In addition, group-care workers who were less extraverted and open were likely to report more child behavior problems than group-care workers with normal levels of extraversion and openness. Finally, no relations were found between agreeableness or conscientiousness and ratings of child behavior. Findings suggest that professionals who work with children are not immune to distortions based on their own personality.     

Stereotypes of singles: are singles what we think?

Four studies examined the accuracy of the single stereotype by comparing perceptions of single and partnered targets with self‐ratings and ratings by others of single and partnered participants. Results revealed that single targets were evaluated more negatively than partnered targets in terms of a wide range of personality characteristics, overall well‐being, and satisfaction with relationships status. These findings were very robust and not qualified by target sex, participant sex, and participant relationship status. In contrast, self‐ratings of single and partnered participants were remarkably similar for all personality characteristics as well as overall well‐being, which was corroborated by ratings of participants by others. However, partnered participants were indeed more satisfied with their relationship status than single participants. When all is considered, the single stereotype is largely inaccurate. Copyright © 2008 John Wiley & Sons, Ltd.     

Age differences in personality traits across cultures: self‐report and observer perspectives

Using self‐report measures, longitudinal studies in the US and cross‐sectional studies from many cultures suggest that the broad factors of Neuroticism, Extraversion, and Openness to Experience decline from adolescence to adulthood, whereas Agreeableness and Conscientiousness increase. Data are inconsistent on the rate of change during adulthood, and on the generalizability of self‐report findings to informant ratings. We analysed cross‐sectional data from self‐reports and informant ratings on the Revised NEO Personality Inventory in Czech (N = 705) and Russian (N = 800) samples. Some curvilinear effects were found, chiefly in the Czech sample; informant data generally replicated self‐reports, although the effects were weaker. Although many of the details are not yet clear, there appear to be pan‐cultural trends in personality development that are consistent with the hypothesis of intrinsic maturation. Copyright © 2004 John Wiley & Sons, Ltd.     

Seeing is Knowing: On the Predictive Accuracy of Self‐ and Informant Reports for Prosocial and Moral Behaviours

Under certain circumstances, well‐known others (so‐called informants) may possess unique insights into targets' personality traits beyond the targets' self‐views. Specifically, as proposed by the self–other knowledge asymmetry model, an incremental predictive ability of informants is most likely for traits and corresponding behaviours that are clearly visible to others and highly evaluative in nature. In two studies, we provide an empirical test of this proposition and extend prior research to one of the most important domains of interpersonal interaction: prosocial and moral behaviours. Specifically, we investigate the unique predictive power of informant reports in trait Honesty–Humility for fairness in the dictator game and dishonesty in a cheating paradigm. Importantly, while both these classes of behaviour are highly evaluative in nature, only fairness is clearly visible to others. Correspondingly, in line with the self–other knowledge asymmetry model, our results reveal unique predictive accuracy of informant reports for fairness. For dishonesty, by contrast, there was no conclusive evidence for incremental predictive power of informant reports. This implies that informants may indeed provide valuable information beyond targets' self‐reports on trait aspects driving fair behaviour, but that targets themselves are their own best experts when it comes to judging trait aspects driving dishonest behaviour. Copyright © 2017 European Association of Personality Psychology     

Multisource Feedback and Leaders' Goal Performance: Moderating effects of rating purpose,rater perspective,and performance dimension

The main purposes of this study are to examine whether multisource feedback ratings predict leaders' organizational goal performance, and whether the relationships are consistent across the two rating purposes (developmental, administrative), two leadership dimensions behaviors (Consideration, Initiating Structure), and three rating perspectives (supervisor, self, and ‘other’ raters, i.e., peers and subordinates). Leaders (n=396) in a large organization in the transportation industry participated in two multisource feedback programs, the first for developmental purposes and the second 8 months later for administrative purposes. Approximately 1 month later, they were rated by their supervisor on their effectiveness in attaining five organizational performance goals (financial, safety, customer satisfaction, employee satisfaction, diversity). Results revealed that both developmental ratings and administrative ratings uniquely predicted leaders' goal performance. However, both leadership dimension and rater perspective moderated these relationships. Leadership behaviors associated with Consideration were stronger predictors of goal performance for supervisor ratings, whereas behaviors associated with Initiating Structure were stronger predictors of goal performance for self and other ratings.     

Agreement of informants on emotional and behavioral problems from childhood to adulthood

Agreement among informants' ratings of children's and adults' internalizing and externalizing problems is moderate. Each informant contributes unique information about an individual's problems. Thus, it has been advocated to obtain ratings from multiple sources in child psychiatry, whereas adult psychiatry relies mostly on self-reports. Longitudinal studies repeatedly assessing children's psychiatric problems from childhood into adulthood and including reports from multiple informants could serve as benchmarks for studies including only selected time points or informants. We examined the development of agreement among informants' ratings of internalizing and externalizing problems using self-, parent, teacher, and partner reports in a longitudinal study with 7 assessment waves spanning an interval of 24 years and covering an age range of 4 to 40 years. The number of informant pairs is 12,059, who rated 1,875 individuals. The results revealed that correlations among informant ratings of internalizing and externalizing problems depend more on the informant pair than on problem type or age group. Second, differences among informants rating internalizing problems typically become larger when individuals get older. Third, when rating themselves, individuals typically report higher scores than do parents, teachers, or partners. These results were consistent for internalizing and externalizing problems and across age groups. The findings indicate that like in child psychiatry, assessment in adult psychiatry may benefit from a shift to multiple informant reports, as different informants' ratings may contain more information than if informants completely agree.     

Assessing Innovation: A 360‐degree appraisal study

This study examines the evaluation of innovation in a wider competency framework and within a 360‐degree rating procedure among managerial‐level job holders. The total sample of 2,979 individuals consisted of 296 target employees and their 318 bosses, 1208 peers, 828 direct reports, and 329 others who provided ratings on a competency framework. The results showed significant differences in innovation‐related competence ratings between different raters. Self‐ratings were significantly lower compared to the overall observer ratings and were correlated only with peer ratings. Different patterns of results were found for the lower and upper quartiles based on self‐ratings. For instance, no correlations were observed between self‐ratings and the ratings of any observers in the group of best self‐rated individuals. Implications for practice and future research in assessment and evaluation of innovation are discussed in conclusion.     

Transparency of Assessment Centers: Lower Criterion‐related Validity but Greater Opportunity to Perform?

Assessment centers (ACs) are popular selection devices in which assessees are assessed on several dimensions during different exercises. Surveys indicate that ACs vary with regard to the transparency of the targeted dimensions and that the number of transparent ACs has increased during recent years. Furthermore, research on this design feature has put conceptual arguments forward regarding the effects of transparency on criterion‐related validity, impression management, and fairness perceptions. This study is the first to examine these effects using supervisor‐rated job performance data as the criterion. We conducted simulated ACs with transparency as a between‐subjects factor. The sample consisted of part‐time employed participants who would soon be applying for a new job. In line with our hypothesis, results showed that ratings from an AC with nontransparent dimensions were more criterion valid than ratings from an AC with transparent dimensions. Concerning impression management, our results supported the hypothesis that transparency moderates the relationship between self‐promotion and job performance, such that self‐promotion in the nontransparent AC was more positively related to job performance than self‐promotion in the transparent AC. The data lent no support for the hypothesis that participants’ perceptions of their opportunity to perform are higher in the transparent AC.     

Aggressive strategies and victimization during adolescence: grade and gender differences,and cross‐informant agreement

The purpose of this study was twofold: (1) to analyze the frequency of physical, verbal, and indirect aggression as well as victimization across two genders and grades and (2) to assess the concordance among different informants. According to the answers provided by 257 fifth and seventh graders (mean age 11.4 and 13.5, respectively) in the Peer Estimated Conflict Behavior Questionnaire, four types of ratings were calculated: same‐ and opposite‐gender rating, rating by both genders, and self‐report. Additionally, teacher ratings were collected. In contrast to earlier findings, boys were found to be directly, as well as indirectly, more aggressive than girls, thus challenging the universality of indirect aggression as a strategy more frequently employed by girls. Consistencies among different informants varied, with peers showing the strongest agreement with each other and self‐views conforming the least to other ratings.Aggr. Behav. 00:1–12, 2005. © 2005 Wiley‐Liss, Inc.     

In the absence of conflicting testimony young children trust inaccurate informants

The present research investigated the nature of the inferences and decisions young children make about informants with a prior history of inaccuracies. Across three experiments, 3‐ and 4‐year‐olds (total N = 182) reacted to previously inaccurate informants who offered testimony in an object‐labeling task. Of central interest was children's willingness to accept information provided by an inaccurate informant in different contexts of being alone, paired with an accurate informant, or paired with a novel (neutral) informant. Experiments 1 and 2 showed that when a previously inaccurate informant was alone and provided testimony that was not in conflict with the testimony of another informant, children systematically accepted the testimony of that informant. Experiment 3 showed that children accepted testimony from a neutral informant over an inaccurate informant when both provided information, but accepted testimony from an inaccurate informant rather than seeking information from an available neutral informant who did not automatically offer information. These results suggest that even though young children use prior history of accuracy to determine the relative reliability of informants, they are quite willing to trust the testimony of a single informant alone, regardless of whether that informant had previously been reliable.     

Cultural differences in athlete attributions for success and failure: the sports pages revisited

Self‐serving biases in attribution, while found with relative consistency in research with Western samples, have rarely been found in Japanese samples typically recruited for research. However, research conducted with Japanese participants to date has tended to use forced‐choice and/or reactive paradigms, with school or university students, focusing mainly on academic performance or arbitrary and/or researcher‐selected tasks. This archival study explored whether self‐serving attributional biases would be shown in the real‐life attributions for sporting performance made by elite Olympic athletes from Japan and Australia. Attributions (N = 216) were extracted from the sports pages of Japanese and Australian newspapers and rated by Australian judges for locus and controllability. It was hypothesized that Australian, but not Japanese, athletes would show self‐serving biases such that they attributed wins to causes more internal and controllable than the causes to which they attributed losses. Contrary to predictions, self‐serving biases were shown to at least some extent by athletes of both nationalities. Both Australian and Japanese men attributed wins to causes more internal than those to which they attributed losses. Women, however, attributed wins and losses to causes that did not differ significantly in terms of locus. All athletes tended to attribute wins to causes that were more controllable than the causes to which losses were attributed. Results are inconsistent with a large body of research suggesting that Japanese do not show self‐serving biases in attribution, and are discussed in the light of differences in methodology, context, and participants that may have contributed to these effects.     

Self‐ratings in Training Programs: An Examination of Level of Performance and the Effects of Feedback

The level of agreement (mean differences and correlations) between self, peer and training staff ratings were examined in this study. The sample consisted of 545 participants who were undertaking a Royal Australian Airforce officer training program. Consistent with previous research there was strong agreement between training staff and peers and weak agreement between self‐ratings and ratings by others (training staff and peers). Accuracy of ratings was examined by (a) comparing the mean ratings of outstanding, average and below‐average performers; and (b) correlating difference scores with a measure of performance. The findings showed that below‐average performers have a less accurate view of themselves compared to outstanding performers. Finally, we examined the effects of negative feedback on self‐perceptions. The analyses indicated that after receiving negative feedback, average performers adjusted their self‐ratings. Various explanations were proposed together with practical implications for training.     

Variability in Personality Expression Across Contexts: A Social Network Approach

The current research investigated how the contextual expression of personality differs across interpersonal relationships. Two related studies were conducted with college samples (Study 1: N = 52, 38 female; Study 2: N = 111, 72 female). Participants in each study completed a five‐factor measure of personality and constructed a social network detailing their 30 most important relationships. Participants used a brief Five‐Factor Model scale to rate their personality as they experience it when with each person in their social network. Multiple informants selected from each social network then rated the target participant's personality (Study 1: N = 227, Study 2: N = 777). Contextual personality ratings demonstrated incremental validity beyond standard global self‐report in predicting specific informants' perceptions. Variability in these contextualized personality ratings was predicted by the position of the other individuals within the social network. Across both studies, participants reported being more extraverted and neurotic, and less conscientious, with more central members of their social networks. Dyadic social network–based assessments of personality provide incremental validity in understanding personality, revealing dynamic patterns of personality variability unobservable with standard assessment techniques.     

RATER SOURCE EFFECTS ARE ALIVE AND WELL AFTER ALL

Recent research has questioned the importance of rater perspective effects on multisource performance ratings (MSPRs). Although making a valuable contribution, we hypothesize that this research has obscured evidence for systematic rater source effects as a result of misspecified models of the structure of multisource performance ratings and inappropriate analytic methods. Accordingly, this study provides a reexamination of the impact of rater source on multisource performance ratings by presenting a set of confirmatory factor analyses of two large samples of multisource performance rating data in which source effects are modeled in the form of second-order factors. Hierarchical confirmatory factor analysis of both samples revealed that the structure of multisource performance ratings can be characterized by general performance, dimensional performance, idiosyncratic rater, and source factors, and that source factors explain (much) more variance in multisource performance ratings whereas general performance explains (much) less variance than was previously believed. These results reinforce the value of collecting performance data from raters occupying different organizational levels and have important implications for research and practice.