A multitrait-multimethod analysis of variance of teachers' ratings of aggression,hyperactivity, and inattention

The convergent and discriminant validities of three teacher rating scale measures of the traits of hyperactivity, aggression, and inattention were explored, using the multitrait-multimethod matrix approach of Campbell and Fiske (1959), as well as an analysis of variance procedure (Stanley, 1961). In the present study teachers rated children from their elementary school classrooms on the above traits. The results provided strong evidence for convergent validity. Data also indicated that these traits can be reliably differentiated by teachers, suggesting that research aimed at better understanding the unique contributions of hyperactivity, aggression, and inattention is warranted. The respective benefits of analyzing multitrait-multimethod matrices by employing the ANOVAprocedure or by using the Campbell and Fiske (1959) criteria were discussed.     

Agreement among student,parent and teacher ratings of school inclusion: A multitrait-multimethod analysis

One of the central objectives of inclusive education, and education in general, is not only to support every students' academic learning, but also their social and emotional development. It therefore is important to identify difficulties in a child's socio-emotional development at school. The current study investigates students' emotional inclusion and social inclusion, as well as students' academic self-concept from four different perspectives using the Perceptions of Inclusion Questionnaire (PIQ). In particular, we analyzed the degree of agreement between teacher, mother, and father ratings with students' self-reports. Moreover, we tested if students' gender and special educational needs (SEN) are predictors for possible bias in parent and teacher reports. Survey participants included 721 Austrian, Grade 4 students from 48 classes. In addition, data from 46 teachers, 466 mother reports, and 375 father reports were included. We assessed the consistency (i.e., agreement) between the different raters by means of multitrait-multimethod analyses, or more precisely, a correlated trait–correlated method minus one (CT-C[M-1]) model. Results of the CT-C(M-1) analyses indicated a rather strong rater bias (i.e., method effects) for all three dimensions of inclusion. However, the consistency for academic self-concept was higher than for emotional and social inclusion. Furthermore, gender and SEN status affected rater bias, particularly for teacher reports. Results suggested that it matters who reports students' emotional inclusion, social inclusion, and academic self-concept, which has methodological and practical implications.     

Relationships between job evaluation ratings and self-ratings of job characteristics

Self-ratings by 308 shop workers of their job variety, autonomy, task identity, and skill challenge correlate significantly with ratings by observers of the jobs' skill requirements. Self-ratings, however, are unrelated to observers' estimates of the physical demands of the jobs or to environmental conditions of the work. The self-ratings and observer measures provide complementary, not substitute, information. Self-ratings can signal person-job dysfunctions, and more objective job measures can identify possible sources of the dysfunctions. Modifications to job characteristics measures are discussed and suggestions are made to begin a new phase of programmatic job characteristics research.     

A comparison of female supervisors in business and government organizations

This exploratory study compared the life style and interpersonal need orientation, leadership style, and perception of the organization structure of female supervisors in business and government organizations. The government group had a significantly higher formalistic and lower sociocentric and personalistic life style orientation; lower needs for expressed control, expressed affection, wanted inclusion, wanted affection, and a higher need for wanted control; were lower on consideration and higher on structure leadership dimensions; and perceived their organization as being more bureaucratic and less collaborative and coordinative than the business group. The findings were discussed in terms of the processes of individual-organization congruity and organizational socialization.     

A comparison of displacement and reinforcer potency for typically developing children

Previous researchers found that individuals with intellectual and developmental disabilities tend to prefer edible over leisure stimuli, although leisure stimuli may still function as reinforcers. We replicated and extended previous research in a 2-part experiment with typically developing children. In Experiment 1, we evaluated 15 children's preference for leisure and edible stimuli. Five of 15 participants preferred edible over leisure stimuli, 3 of 15 participants preferred leisure over edible stimuli, and the remaining 7 of 15 participants did not show a preference for a stimulus class. In Experiment 2, we compared the reinforcer potency of the top-ranked stimulus from each class with 7 of the 8 participants who showed displacement of one stimulus class. Four of 7 participants allocated more responding to the task associated with the top-ranked stimulus and 3 of 7 participants showed no differences in responding to the task regardless of the stimulus rank.     

Environments as reinforcer systems in the study of job satisfaction

This paper examined differences in the degree to which fifteen specific sources of job satisfaction were related to and predictive of the overall satisfaction of department chairmen in the six model environments proposed by Holland (1973). The fifteen specific sources of job satisfaction were regressed on overall satisfaction, separately for chairmen in each environment, through the use of incremental stepwise multiple regression procedures. The results indicated that the overall satisfaction of chairmen in these six environments was differentially related to unique weightings and combinations of the fifteen predictor variables. It was concluded that environments, composed of essentially similar groups of people in different organizations, serve as job satisfaction reinforcer systems in a manner similar to organizations as illustrated by Dawis.     

Understanding the latent structure of job performance ratings

This study quantified the effects of 5 factors postulated to influence performance ratings: the ratee's general level of performance, the ratee's performance on a specific dimension, the rater's idiosyncratic rating tendencies, the rater's organizational perspective, and random measurement error. Two large data sets, consisting of managers (n = 2,350 and n = 2,142) who received developmental ratings on 3 performance dimensions from 7 raters (2 bosses, 2 peers, 2 subordinates, and self) were used. Results indicated that idiosyncratic rater effects (62% and 53%) accounted for over half of the rating variance in both data sets. The combined effects of general and dimensional ratee performance (21% and 25%) were less than half the size of the idiosyncratic rater effects. Small perspective-related effects were found in boss and subordinate ratings but not in peer ratings. Average random error effects in the 2 data sets were 11% and 18%.     

Describing organizations as reinforcer systems: A new use for job satisfaction and employee attitude surveys

Job satisfaction data on managerial personnel were used to illustrate how organizations can be differentiated and described as reinforcer systems. Twenty-seven specific satisfaction scales were regressed on overall job satisfaction, separately for each of five groups of managers from five organizations. For all groups, overall satisfaction was found to be determined mainly by scales concerning challenge of the job and prospects of one's chosen career. Application of decremental stepwise multiple regression procedures to obtain minimum-variable-set equations resulted in different, though overlapping, sets of variables in the equations for the different organizations. These results suggest a way of describing organizations as reinforcer systems, in terms of the specific determinants of overall satisfaction for the organization.     

Effects of analyst training and amount of available job related information on job analysis ratings

The present study investigated the effects of job analysis training and the amount of information available to the job analyst on job analysis ratings. Forty subjects were asked to complete the Position Analysis Questionnaire (PAQ) and the Job Components Inventory (JCI) for the position of College Resident Assistant. Results indicated that when using the PAQ as the job analysis instrument, there was no significant effect for either training or the amount of information received by the job analyst. For the JCI, the results indicated that trained analysts were more reliable and accurate than untrained analysts. Also, it was found that the more information received by the job analysts, the more reliable and accurate were their JCI ratings.     

The value of relative-time-spent ratings in task-oriented job analysis

Recent studies have called for the abandonment of the relative-time-spent scale in task inventories. This recommendation is based on findings that the job profile created with the scale data is highly correlated with the profile created from a much simpler “Do you perform this task?” checklist. We examined this issue using 3 inventories and 42 jobs (N=2252). Profile correlations were computed on only the tasks actually performed by incumbents to avoid possibly inflated rs due to including irrelevant tasks. The specificity of task inventory items was proposed as an explanation for the high correlation between the two job profiles. Specificity of items was examined by looking at both the type (job duties versus tasks) and the amount (number of items in job profile and average number of items relevant to each job) of items used in the inventory. Correlations between time spent and checklist profiles were in the .80's and .90's regardless of the number of irrelevant tasks or the specificity of tasks. We agree with previous military research and conclude that the relative-time-spent scale has limited incremental utility beyond a dichotomous checklist.     

A comparison of multiple reinforcer assessments to identify the function of maladaptive behavior

Functional analysis results indicated that maladaptive behaviors displayed by a 25-year-old man with profound mental retardation were maintained by attention from caregivers and possibly, to a lesser degree, by access to tangible items. A concurrent-schedules procedure was then used to study the relative reinforcing value for maladaptive behavior of attention versus tangible items. Results of the concurrent-schedules assessment and subsequent functional communication training indicated that preference for attention versus access to a tangible object varied.     

Interrater reliability and agreement of performance ratings: A methodological comparison

This paper demonstrates and compares methods for estimating the interrater reliability and interrater agreement of performance ratings. These methods can be used by applied researchers to investigate the quality of ratings gathered, for example, as criteria for a validity study, or as performance measures for selection or promotional purposes. While estimates of interrater reliability are frequently used for these purposes, indices of interrater agreement appear to be rarely reported for performance ratings. A recommended index of interrater agreement, theT index (Tinsley & Weiss, 1975), is compared to four methods of estimating interrater reliability (Pearsonr, coefficient alpha, mean correlation between raters, and intraclass correlation). Subordinate and superior ratings of the performance of 100 managers were used in these analyses. The results indicated that, in general, interrater agreement and reliability among subordinates were fairly high. Interrater agreement between subordinates and superiors was moderately high; however, interrater reliability between these two rating sources was very low. The results demonstrate that interrater agreement and reliability are distinct indices and that both should be reported. Reasons are discussed as to why interrater reliability should not be reported alone.This paper is based, in part, on a thesis submitted to East Carolina University by the second author. Portions of this study were presented at the American Psychological Association meeting in New Orleans, LA, August, 1989. The authors would like to thank Michael Campion and two anonymous reviewers for their comments on earlier drafts of this paper.