Back to basics: Percentage agreement measures are adequate, but there are easier ways

Percentage agreement measures of interobserver agreement or "reliability" have traditionally been used to summarize observer agreement from studies using interval recording, time-sampling, and trial-scoring data collection procedures. Recent articles disagree on whether to continue using these percentage agreement measures, and on which ones to use, and what to do about chance agreements if their use is continued. Much of the disagreement derives from the need to be reasonably certain we do not accept as evidence of true interobserver agreement those agreement levels which are substantially probable as a result of chance observer agreement. The various percentage agreement measures are shown to be adequate to this task, but easier ways are discussed. Tables are given to permit checking to see if obtained disagreements are unlikely due to chance. Particularly important is the discovery of a simple rule that, when met, makes the tables unnecessary. If reliability checks using 50 or more observation occasions produce 10% or fewer disagreements, for behavior rates from 10% through 90%, the agreement achieved is quite improbably the result of chance agreement.     

Apparatus for collecting voice chronography data

The conversion of speech into on-off pulse patteras representing talk spurts and pauses is a major tool used by psychologists in analyzing interactive communications behavior. We describe a conversion method and a means for interfacing its output with a computer, which allows the collection of large amounts of speech data from one or more sources. The major improvement over prior art is the ability to reject cross talk between neighbor speakers while permitting the detection of simultaneous speech. In the evaluation of the performance of the apparatus, no errors were detected for single-party speaking. Infrequent small errors occurred in the detection and duration of talk spurts and pauses during two- and three-party speaking, but these would probably be acceptable in most applications.     

Word and nonword superiority effects in a letter detection task

Subjects searched for predesignated consonant letters embedded in strings of consonants, consonants and vowels, or consonants and numbers. In Experiments 1-3, detection was quicker in the consonant-vowel and consonant-number strings than in the consonant strings. Apparently, vowels and numbers were less confusable distractors than are nontarget consonants. Experiment 4 tested whether psychophysical or categorical information about letters and numbers enabled subjects to process consonant-vowel and consonant-number strings more quickly. Results indicated that psychophysical characteristics of target and distractor letters mediated both word and nonword superiority effects.

     

Reflection-impulsivity in normal and behavior-disordered children

The primary index of reflection- impulsivity is Kagan's Matching Familiar Figures Test (MFF), which yields both a latency and an error score. To evaluate further the construct validity of these measures, both the latency and error scores of 9- and 15-year-old normal and acting-out behavior-disordered children were compared. Young behavior-disordered children were found to be more impulsive than the other groups on the MFF error measure. No differences occurred on the MFF latency measure. This result is consistent with previous findings with normal children in supporting the construct validity of the MFF error score and raising questions about the construct validity of the MFF latency score. Findings were inconsistent with Kagan's assertion that normal children become more reflective with age. The older behavior-disordered children were more reflective on the error measure than the younger behavior-disordered children and had equivalent error scores to both age groups of normals. This finding suggested a lag in the development of reflection in behavior-disordered children.The authors wish to thank Dr. Kenneth C. Molkner for his valuable assistance in obtaining subjects and for his kind support.     

Two kinds of adaptation in the constancy of visual direction and their different effects on the perception of shape and visual direction

Adaptation in the constancy of visual direction can be obtained under two radically different conditions, called eye-movement adaptation and field adaptation. Adaptation resulting from these conditions and from a “normal” condition was measured with a newly developed estimation test. Eye-movement adaptation was found to cause an alteration of compensatory eye movements. It apparently consists of a changed evaluation of eye movements, as demonstrated by two different pointing tests. A form test where the shape of a large oblong is set to look square also confirmed this interpretation. After field adaptation, a pointing test did not register a change, but an adaptation effect could be measured with a forward direction test. This test and a square test where no eye movements were permitted proved to be specific to field adaptation; they measured no effect after eye-movementadaptation. The normal adaptation condition Was apparently equivalent to the eye-movement adaptation condition. Its effect could be measured only with a pointing test. When we changed the normal adaptation condition so that frequent saccades were made during head turning, strong effects were measured with the two tests that were specific to field adaptation.     

Stimulus and response factors in mirror-image discrimination

Two choice-reaction time studies assessed the influence of stimulus-response mapping, stimulus complexity, and stimulus alignment on adults’ discrimination of mirror-image and nonmirror-image stimulus pairs. Half the subjects in Experiment 1 were instructed to treat nonmiiTor pairs as “same” and mirror pairs as “different”; the other half responded in the opposite manner. The first group responded more quickly to nonmirror pairs, while the second group responded more quickly to mirror pairs. This result, which held for horizontal stimuli (side by side) as well as for vertical stimuli (one above the other), confirms the importance of experiential factors in mirror-image “confusions. “ In Experiment 2. stimuli were drawn from a population of patterns whose complexity could be objectively defined. In general, the more complex the pattern, the slower the response and complexity seemed to influence the qualitative nature of pattern processing. In both experiments, subjects responded more quickly to horizontal stimuli than to vertical stimuli.     

Startbox-goalbox confinement durations as determinants of self-punitive behavior

Two levels of confinement duration (5 or 60 sec) in either startbox or goalbox were factorially combined with presence or absence of shock during extinction of a runway escape response in a self-punitive paradigm under a spaced-trials procedure. All groups were equated for amount and temporal spacing of handling before and after each trial. Shocked rats were more resistant to extinction than nonshocked rats only when startbox confinement was short. When startbox confinement was long, shock facilitated extinction. Long goalbox confinement enhanced running speed for shocked rats, but only in the presence of shock. It was suggested that duration of startbox confinement affects strength of conditioned fear, with long confinement leading to its extinction.     

Wilhelm Dilthey: Forerunner of humanist social thought

Humanist social thought is as a meadow in the forest of positivist science. Much of this space was cleared by Wilhelm Dilthey, not only through his attack on the fundamental assumptions of positivism, but also through his formulation of a critical method by which the works of free human consciousness could be understood. The first tenet of positivism is that the world is made up of ‘out there’ objectively knowable ‘facts’. Dilthey undercut this notion by asserting that the subject matter of the human studies was not mere facts of nature, but rather objectified expressions of the human mind. The second central assumption of positivism is that these facts are explainable or determined by general causal laws. In contrast, Dilthey asserted that, while we can explain the natural world, human action must be understood through an interpretive rather than a causal logic. In demonstrating and specifically describing such an interpretive procedure, Dilthey provided an epistemological and methodological grounding fur a humanistic science of the person and of the social world. His ideas illuminate the works even of his critics and his influence, though largely unacknowledged, continues to be widespread in all the human studies.     

Alcohol-induced changes in contrast sensitivity following high-intensity light exposure

Relatively low doses of alcohol produced large, significant, dose-related increases in the time required to recover foveal contrast sensitivity following bright light exposure. Nine subjects participated in a double-blind experiment involving three dose levels of alcohol (including placebo). The luminance parameters of the test were comparable to those encountered in practical situations such as driving. The alcohol-induced delay in glare recovery is probably retinal and lasts for several hours after drinking.