A graphical judgmental aid which summarizes obtained and chance reliability data and helps assess the believability of experimental effects

Interval by interval reliability has been criticized for "inflating" observer agreement when target behavior rates are very low or very high. Scored interval reliability and its converse, unscored interval reliability, however, vary as target behavior rates vary when observer disagreement rates are constant. These problems, along with the existence of "chance" values of each reliability which also vary as a function of response rate, may cause researchers and consumers difficulty in interpreting observer agreement measures. Because each of these reliabilities essentially compares observer disagreements to a different base, it is suggested that the disagreement rate itself be the first measure of agreement examined, and its magnitude relative to occurrence and to nonoccurrence agreements then be considered. This is easily done via a graphic presentation of the disagreement range as a bandwidth around reported rates of target behavior. Such a graphic presentation summarizes all the information collected during reliability assessments and permits visual determination of each of the three reliabilities. In addition, graphing the "chance" disagreement range around the bandwidth permits easy determination of whether or not true observer agreement has likely been demonstrated. Finally, the limits of the disagreement bandwidth help assess the believability of claimed experimental effects: those leaving no overlap between disagreement ranges are probably believable, others are not.     

TEACHING RETARDED WOMEN A CLOTHING SELECTION SKILL USING COMMUNITY NORMS

Five institutionalized retarded women were taught a clothing selection skill to coincide with popular fashion. Normative data concerning popular styles of color-coordinated clothing were collected through observations of women's apparel in a local community. Using a puzzle simulation of a woman with alternative pieces of colored clothing, color coordination training was conducted using modelling, instructions, practice, praise, and feedback to teach popular selections of color combinations. A generalization training component with actual clothing was also included. Training was accompanied by large increases in percentage of popular color selections with the puzzle and actual clothing. The increases maintained over a seven- to 14-week followup period. Time efficiency measures supported the use of the simulation procedure as opposed to total reliance on actual clothing. The results underscore the utility of local norms in community preparation programs and the efficacy of instructional procedures for severely and profoundly retarded persons as part of their participation in the deinstitutionalization movement.     

A study of the “central life interests” and “work attachments” of male and female workers

This paper reports the results of a survey undertaken to investigate the overall central life interests and attachments to work of a sample of 1112 male and female employees of six Southern California manufacturing firms. The major conclusions suggested by this study are three. First, male employees have a slight tendency to be more work oriented in their overall central life interests than female employees. Second, female employees tend to be more strongly attached to extrinsic work features than male employees. Third, male-female differences in occupational status and perceived mobility opportunities probably account for gender differences in overall central life interests but these variables do not account for the tendency for women to be more strongly attached than men to extrinsic work features.     

Multidimensional scaling of nominal data: The recovery of metric information with alscal

Multidimensional scaling has recently been enhanced so that data defined at only the nominal level of measurement can be analyzed. The efficacy of ALSCAL, an individual differences multidimensional scaling program which can analyze data defined at the nominal, ordinal, interval and ratio levels of measurement, is the subject of this paper. A Monte Carlo study is presented which indicates that (a) if we know the correct level of measurement then ALSCAL can be used to recover the metric information presumed to underlie the data; and that (b) if we do not know the correct level of measurement then ALSCAL can be used to determine the correct level and to recover the underlying metric structure. This study also indicates, however, that with nominal data ALSCAL is quite likely to obtain solutions which are not globally optimal, and that in these cases the recovery of metric structure is quite poor. A second study is presented which isolates the potential cause of these problems and forms the basis for a suggested modification of the ALSCAL algorithm which should reduce the frequency of locally optimal solutions.     

Relating tests given to different samples

Suppose a collection of standard tests is given to all subjects in a random sample, but a different new test is given to each group of subjects in nonoverlapping subsamples. A simple method is developed for displaying the information that the data set contains about the correlational structure of the new tests. This is possible to some extent, even though each subject takes only one new test. The method uses plausible values of the partial correlations among the new tests given the standard tests in order to generate plausible simple correlations among the new tests and plausible multiple correlations between composites of the new tests and the standard tests. The real data example included suggests that the method can be useful in practical problems.     

Measuring nominal scale agreement between a judge and a known standard

Two designs for comparing a judge's ratings with a known standard are presented and compared. Design A pertains to the situation where the judge is asked to categorize each ofN subjects into one ofr (known) classes with no knowledge of the actual number in each class. Design B is employed when the judge is given the actual number in each class and is asked to categorize the individuals subject to these constraints. The probability distribution of the total number of correct choices is developed in each case. A power comparison of the two procedures is undertaken.     

Improving the efficiency and effectiveness of interactively selected MDS data designs

Two revisions of interactive MDS data selection procedures are presented. One revision improves the estimates of the MDS parameters by adding an analysis of the volume of the spatial coordinates of stimuli. Frames of stimuli augmented by an analysis of volume should more nearly surround the swarm of stimulus points. The second revision, based on randomly ordering the list of stimuli, permits more efficient data designs to be selected by reducing the number of judgments collected but never analyzed.This investigation was supported in part by a Research Scientist Development Award (KO2-DA00017), research grants (DA01070 and MH24149) from the U.S. Public Health Service, and a research and instruction award (IUC 2-47622-1990) from the University of California.     

On the precision of a euclidean structure

This paper is concerned with the development of a measure of the precision of a multidimensional euclidean structure. The measure is a precision index for each point in the structure, assuming that all the other points are precisely located. The measure is defined and two numerical methods are presented for its calculation. A small Monte Carlo study of the measure's behavior is performed and findings discussed.The authors are indebted to Bert F. Green, Jr., Ronald Helms, Andrea Sedlak, and three anonymous reviewers for their valuable comments on earlier drafts of this paper.     

A durable recording and feedback system

A durable low-cost data-recording and feedback system is described. Data transcribed with a special pen onto a plastic sheet can be removed only with two solvents. These data can be displayed for feedback purposes without fear of destruction.