Measurement Equivalence and Multisource Ratings for Non-managerial Positions: Recommendations for Research and Practice

ABSTRACT: The present investigation applies a comprehensive sequence of confirmatory factor analysis tests (Vandenberg and Lance, Organisational Research Methods, 3, 4–69, 2000) to the examination of the measurement equivalence of self, peer, and supervisor ratings of non-managerial targets across several performance dimensions. Results indicate a high degree of measurement equivalence across rater sources and performance dimensions. The paper illustrates how this procedure can identify very specific areas of non-equivalence and how the complexity of a multisource feedback system may be represented using such procedures. Implications of these results and recommendations for both research and practice are offered.     

Internationalizing 360-Degree Feedback: Are Subordinate Ratings Comparable?

ABSTRACT: This study addressed whether 360-degree feedback ratings made by subordinates are equivalent across national cultures (Great Britain, Hong Kong, Japan, U.S.). Employees ratings of their leaders were collected via a companys existing 360-degree feedback survey. A two-factor baseline model was developed using exploratory factor analysis and tested across cultures utilizing confirmatory factor analysis techniques. Results revealed differences in factor form, suggesting that the constructs underlying the survey and their relationship to the survey items differed across cultures. These results emphasize the need for multinational companies to use caution when transporting human resources practices, such as 360-degree feedback, to international locations.The author would like to express appreciation to Alice Stuhlmacher for her valuable comments on earlier versions of this article. This research was conducted as part of the authors doctoral dissertation. Portions of this article were presented at the 17th Annual Conference of the Society for Industrial and Organizational Psychology, Toronto, Ontario, Canada, April 2002.     

Managerial Experience and the Measurement Equivalence of Performance Ratings

ABSTRACT: Establishing the measurement equivalence of instruments is a prerequisite to making meaningful comparisons between individuals or within individuals over time. Whereas previous research has investigated the effects of rater characteristics on the measurement equivalence of performance ratings, the current study investigated a ratee characteristic—ratee job experience. Using confirmatory factor analysis and item response theory methods with replication, the measurement equivalence of supervisor ratings of 7,200 managers with differing levels of managerial experience was assessed. Overall, results indicated a high degree of measurement equivalence suggesting that meaningful comparison may be made across ratees with different levels of job experience.     

Associative symmetry in the pigeon after successive matching-to-sample training

If an organism is explicitly taught an A→B association, then might it also spontaneously learn the symmetrical B→A association? Little evidence attests to such “associative symmetry” in nonhuman animals. We report for the first time a clear case of associative symmetry in the pigeon. Experiment 1 used a successive go/no go matching‐to‐sample procedure, which showed all of the training and testing stimuli in one location and intermixed arbitrary and identity matching trials. We found symmetrical responding that was as robust during testing (B→A) as during training (A→B). In Experiment 2, we trained different pigeons using only arbitrary matching trials before symmetry testing. No symmetrical responding was found. In Experiment 3, we trained other pigeons with only arbitrary matching trials and then tested for symmetry. When these pigeons, too, did not exhibit symmetrical responding, we retrained them with intermixed identity and arbitrary matching trials. Less robust symmetrical responding was obtained here than in Experiment 1. Collectively, these results suggest that identity matching may have to be learned concurrently with arbitrary matching from the outset of training for symmetry to emerge.     

Stability of functional equivalence and stimulus equivalence: effects of baseline reversals

Functional equivalence and stimulus equivalence classes were established, reversed, and tested for stability with college students. Functional stimulus classes were established using a task in which students were trained to say nonsense words in the presence of arbitrarily assigned sets of symbols. Computer-controlled speech-recognition technology was used to record and analyze students' vocal responses for accuracy. After the establishment of stimulus classes was demonstrated with a transfer-of-function test, the effects of reversing selected baseline simple discriminations were assessed during an additional transfer-of-function test and a follow-up test that occurred several weeks later. With the same students, stimulus equivalence classes were established and demonstrated with computerized matching-to-sample procedures. The effects of reversing selected baseline conditional discriminations also were assessed during a postreversal equivalence test and a follow-up test. Both functional stimulus classes and stimulus equivalence were sensitive to contingency reversals, but the reversals with stimulus equivalence closses affected stimulus class organization whereas reversals with functional stimulus classes did not. Follow-up performances were largely consistent with the original baseline contingencies. The similarities and differences between stimulus equivalence and functional equivalence are related to the specific contingencies that select responding in the presence of the stimuli that form the classes.     

Contextual control of stimulus generalization and stimulus equivalence in hierarchical categorization

The purpose of this study was to determine whether hierarchical categorization would result from a combination of contextually controlled conditional discrimination training, stimulus generalization, and stimulus equivalence. First, differential selection responses to a specific stimulus feature were brought under contextual control. This contextual control was hierarchical in that stimuli at the top of the hierarchy all evoked one response, whereas those at the bottom each evoked different responses. The evocative functions of these stimuli generalized in predictable ways along a dimension of physical similarity. Then, these functions were indirectly acquired by a set of nonsense syllables that were related via transitivity relations to the originally trained stimuli. These nonsense syllables effectively served as names for the different stimulus classes within each level of the hierarchy.     

Fonagy and Freud. Psychological versus psychic reality

Peter Fonagy and Mary Target present their Playing with reality theory as a developmental theory centred on the concept of psychic reality. This paper compares Fonagy and Target's use of the concept of psychic reality with Freud's original concept. It is argued that the concept of psychic reality has been redefined from delineating a psychic reality stemming from the unconscious to denoting a kind of conscious or preconscious psychological reality characterized by an experience of equality between the internal and the external worlds. The theoretical discussion is illustrated by being applied to eating disorder pathology, which by Fonagy and colleagues is described as associated with thought processes characterized by psychic reality.     

Measurement Equivalence/Invariance of the Abusive Supervision Measure Across Workers From Taiwan and the United States

Growing international research interest in negative-leadership behaviors prompts the need to examine whether measures of ineffective leadership developed in the United States are equivalent across countries outside the United States. B. J. Tepper's (2000) abusive supervision measure has been used widely inside and outside the United States and merits research attention on its construct equivalence across different cultural settings. The authors conducted a series of multigroup confirmatory factor analyses to investigate the measurement equivalence of this measure across Taiwan (N = 256) and the United States (N = 389). Configural invariance was established, suggesting that both U.S. and Taiwanese samples perceive abusive supervision as a single-factor concept. Furthermore, the establishment of partial metric invariance and partial scalar invariance suggests that the abusive supervision measure is applicable to crosscultural comparisons in latent means, construct variance, construct covariances, and unstandardized path coefficients with the caution that workers from different cultures calibrate their responses differently when answering some items.     

Work engagement and workaholism: comparing the self-employed and salaried employees

This study among a Dutch convenience sample of self-employed individuals (n = 262) and salaried employees (n = 1900) tested to what extent workaholism and work engagement relate to self-reported work performance. After controlling for measurement inequivalence, results of structural equation modeling showed that the self-employed score higher on engagement and working excessively then employees, but not on working compulsively. In addition, work engagement related positively to task performance and innovativeness for both groups. However, engagement only related to contextual performance (performance beyond role requirements) for employees. Workaholism had positive and negative relationships with self-reported performance. Working excessively related positively to innovativeness for both groups, and to contextual performance for the self-employed. Working compulsively suppressed this positive relationship between excessive working and innovativeness in both groups, and between excessive working and contextual performance for the self-employed. In contrast to our expectations, working compulsively related positively to contextual performance for employees.     

Psychometric evaluation of the Basic Traits Inventory in the multilingual South African environment

The objective of this study was to evaluate the psychometric properties of the BTI for all the official language groups in South Africa. Item bias analysis was performed by means of Rasch analysis for a sample of 105 342 respondents with representation of each of the 11 language groups. Reliability values were good for all the language groups and bias was revealed in only 12 of the 193 items (6.2%) through item response modelling. Based on these results, the BTI may be an appropriate measure of personality for all the different language groups in South Africa.