Measures of interobserver agreement: Calculation formulas and distribution effects

Seventeen measures of association for observer reliability (interobserver agreement) are reviewed and computational formulas are given in a common notational system. An empirical comparison of 10 of these measures is made over a range of potential reliability check results. The effects on percentage and correlational measures of occurrence frequency, error frequency, and error distribution are examined. The question of which is the best measure of interobserver agreement is discussed in terms of critical issues to be considered     

Factors associated with parent labeling of children referred for conduct problems

Clinicians have suggested that in some cases normal children have been inappropriately labeled as deviant by their parents and taken to psychological clinics for treatment. Reasons given for such inappropriate labeling have included factors such as the parents' marital distress and intolerance of normal child behavior. This study provided an empirical examination of the appropriateness or inappropriateness of parent labeling of 5–12-year-old children referred by their parents for treatment of conduct problems. The definition of inappropriate labeling derived by the present investigators required that the relationship between the child's actual behavior and the parent's negative label be so imperfect as to raise questions about the justification for the label, and that some identifiable factors other than the child's behavior account for the negative label. Three parent factors evaluated for their contribution to the parent negative label were marital distress, parent negative behavior toward the child, and parent distress about child deviant behavior. The statistical model of hierarchical multiple regression permitted analysis of this definition. The association found between observed child deviant behavior and the parent negative label suggested that parents of this sample perceived their children reasonably accurately. The three parent factors were found to be negligibly related to the parent label. Therefore, the conditions for inappropriate labeling of the children by the parents of this sample were not met.     

Speech prosody in Broca's aphasia

Because speech prosody is thought to be impaired in Broca's aphasia, we conducted three experiments using Broca's aphasics and nonneurological control subjects in order to determine to what extent this statement is accurate. Using three acoustic measures of speech prosody—sentence-final fundamental frequency (F₀) fall, F₀ declination, and sentence-final lengthening—we found that some aspects of prosody were spared while others were abnormal. All Broca's aphasics, regardless of degree of impairment, exhibited sentence-final F₀ fall. F₀ declination was present in simple sentences but was absent over longer domains; moreover, the amount of declination was correlated with the degree of linguistic impairment. Sentence-final lengthening was clearly absent in Broca's aphasia; in fact, sentence-terminal words were actually shorter than their sentence-initial and medial counterparts.     

The relationship between parent-youth interaction and delinquency in group homes

Interactions between the youths and group home parents in 10 group homes were directly observed during a 2-hour session in each home. These observations were compared to questionnaire measures of the youths' self-reported delinquency while in the group home, and their evaluation of the group home program. The results indicated that over the 10 homes there was a strong inverse correlation between mean self-reported delinquency and the average amount of time youths spent taking to (r _s =–.95)and in proximity to (r _s =–.81)their group home parents. Talking to group home parents also showed a positive correlation with the youths' evaluation of their group home program. Similar trends were found when the correlations were computed over individual youths, although the magnitudes were reduced. These data may have implications for our understanding of the relationship between parenting behaviors and delinquency, as well as for the improvement of group-home treatment programs.This research was supported by Grant MH20030 and by a postdoctoral research fellowship (Grant MH15200, awarded to the first author) from the National Institute of Mental Health (Center for Studies in Crime and Delinquency). These grants were made to the Bureau of Child Research and the Department of Human Development, University of Kansas. The authors wish to thank Susan Whitely for her statistical consultation, and Jay Atwater, Carol Cramer, and Lynette Forbes for their help in the data analysis. Appreciation is also extended to the many teaching-parents whose cooperation made the research possible.     

OUTCOME EVALUATION OF BEHAVIORAL PARENT TRAINING AND CLIENT-CENTERED PARENT COUNSELING FOR CHILDREN WITH CONDUCT PROBLEMS1

The central question addressed was, how effective is parent training in reducing conduct problems in children in comparison to client-centered parent counseling? A secondary question was the relative effectiveness of the two treatment groups in comparison to a wait control group that when untreated during the 8-week period of treatment provided the other groups. Families of 36, 5- to 12-year-old conduct problem children were screened and assigned at random to treatment groups, but wait control group assignment depended upon therapist availability. Supervised graduate student therapists conducted 10 treatment sessions for each family. Parent reports and paper and pencil tests of child deviance and parent satisfaction showed a superior outcome for behavioral over the client-centered and wait control groups, and no differences between the latter two groups. At follow-up there was no maintenance of this superiority. Home observation data showed no advantage of behavioral over client-centered treatment, and these two groups did not improve significantly more than the wait control group. These results were discussed in the light of possible interactions between treatment and measurement, and methodological and sampling differences between this and other studies.     

The effect of spatial orientation on the perception of moving tactile stimuli

Previous studies have shown that the perception of spatial patterns, such as letters, presented to the hand is affected by the spatial orientation of the hand. The present study investigated how the perception of direction of motion across the fingerpads changes with the position of the hand in space. The moving stimuli were generated on two displays. In one condition, the displays were placed horizontally in front of the subject, with the subject’s thumb (target site) and index finger (nontarget site) placed flat on the displays. In a second condition, the displays were vertically oriented and gripped between the thumb and index finger. Using a selective-attention paradigm in which subjects are instructed to respond only to the direction of motion at the target site, performance was still affected by the direction of motion at the nontarget site. Changing the orientation of the displays changed the effectiveness of the nontarget in interfering with the identification of the target movement. Nontarget stimuli that produced no interference in the horizontal orientation did so in the vertical, and vice versa. It appears that subjects are not using the local direction of movement across the fingerpads to judge the relative direction of movement at the two sites; rather, they are using the external direction of movement.     

At-entry differences in part-time and full-time employees

Research consistently shows part- and full-time workers to be different on a variety of characteristics such as job satisfaction, organizational commitment and intentions to leave the organization. This study challenges the commonly accepted argument that these differences are due to the different structural conditions of work that are faced by part- and full-time workers. The possibility that these differences are the consequence of at-entry differences in dispositions and orientations to work is tested on a sample of part- and full-time registered nurses who had just begun employment. With one exception the part-and full-time employees were found to be the same on fourteen at-entry characteristics. These results strongly suggest that it is the structural conditions of work that produce the often observed differences found between part- and full-time workers.An earlier version of this paper was presented at the 1991 Midwest Sociological Society Meetings, Des Moines, Iowa, April 11–14. Support for the original data collection came from the Division of Nursing of the Health Resources and Services Administration (R01 NU01050-01A1). Joanne McCloskey is to be thanked for allowing us to use this data set.     

Metacognition and mathematics strategy use

The role of children's metacognitive knowledge in their mathematics strategy use was studied by a longitudinal examination of second graders' effort attributions, metacognition for mathematics, and strategy use while solving mathematics problems. Children's correct use of retrieval, internal and external strategies, and the prevalence of strategy use were assessed in September and the following January. Effort attributions for success and failure were also assessed at both points in time. In January, metacognitive knowledge about mathematics strategies was measured. Second graders possess metacognitive knowledge about mathematics strategies, and this knowledge is correlated most strongly with the tendency to use internal strategies in September and correct internal strategy use in September. Effort attributions measured at both timepoints were significantly related to metacognition. Effort attributions in January also correlated with the tendency to use internal strategies in January. In general, the results are consistent with self-system theories, which posit that metacognition, motivation, and strategy use work together to promote learning.     

Very long-term memories of a salient preschool event

Children who attended either a younger (mean age = 3frac12;years) or an older (mean age = 4frac12; years)preschool classroom were interviewed twice about an emergency school evacuation in response to a fire alarm. All children were able to answer some memory questions 2 weeks after the evacuation occurred, but the memory narratives produced by older preschool children showed a more refined knowledge of the temporal and causal sequence of events. Seven years later, memory of the fire alarm was reassessed. Only those preadolescents who had been in the older preschool group at the time of the alarm showed convincing evidence of long-term memory. The offset of childhood amnesia appears to be influenced by developmental changes in cognitive processing.