AN OBJECTIVE EVALUATION OF A BEHAVIOR MODELING TRAINING PROGRAM

Past research on the effects of behavior modeling training has rarely focussed on actual changes in job performance. This paper describes a study in which results of a behavior modeling training program for sales representatives were evaluated in relation to effects on the sales performance of the participants. Sales associates selling large appliances, radios, and television sets for a large chain retailer in seven stores in one metropolitan area participated in a behavior modeling training program designed to improve their sales effectiveness. Before and after comparisons were made in their sales records with similar sales associates in seven matched stores who participated in other kinds of sales training during the same period. Sales representatives who received the behavior modeling training increased their sales by an average of 7% during the ensuing six-month period, while their counterparts in the control group stores showed a 3% decrease in average sales. The extra effort entailed in obtaining the needed sales records seemed to be worthwhile in providing convincing evidence of the value of the training.     

AN ANALYSIS OF MULTIPLE MISPLACED PARENTAL SOCIAL CONTINGENCIES1

This study analyzed the training of a mother to modify five subclasses of her attention to her young child's noncompliance with instructions, and also displayed the changes in her child's behavior correlated with these events. Training in four subclasses consisted of teaching the mother to withhold various forms of social attention to her daughter's undesired behavior; training in the fifth subclass involved introduction of a brief room-timeout procedure for noncompliance. The effectiveness of the parent-training procedure, consisting of initial instructions and daily feedback, was demonstrated through a multiple-baseline design across the five subclasses of parent behavior. Sequential decreases in the first three subclasses of the mother's social attention to undesired child behavior resulted in incomplete improvements in some child responses; however, a decrease in the fourth subclass resulted in a significant increase in undesired child behavior. Complete remediation of all child behaviors was achieved following the training of a timeout procedure for noncompliance. Postchecks conducted up to 16 weeks later showed that these effects were durable.     

EVALUATION OF AN INTERVIEWEE SKILLS TRAINING PROGRAM IN A NATURAL FIELD EXPERIMENT

The purpose of this study was to provide an evaluation of an interviewee skills training program in a field setting using a broad sample of interviewers, jobs, and candidates. The training program ( n = 158) derived its content from previous literature and used multiple instructional techniques. An experimental design was employed with a self-study (placebo control) group ( n = 140), random assignment, and many levels of evaluation criteria. Measures exhibited acceptable reliability, and statistical power was high for all analyses. Results indicated that class members responded positively to the program and demonstrated substantial learning. Interviewer evaluations of behavior failed to distinguish between experimental groups, however, and no differences were observed in job offers. Furthermore, no differences were observed between study participants and nonparticipants ( n = 174) in terms of interview behavior and job offers. This study illustrates the danger of relying solely on reaction or learning criteria in the evaluation of training programs. Several explanations are offered for the findings.     

APPLICATION OF A SOCIAL SKILLS TRAINING PROGRAM IN THE MODIFICATION OF INTERPERSONAL DEFICITS AMONG RETARDED ADULTS: A CLINICAL REPLICATION

Six mentally retarded adults, equally divided into two treatment groups, were provided with individualized social skills training programs. Treatment, evaluated via a multiple-baseline design strategy, was sequentially and cumulatively applied across target behaviors over a four-week intervention period. Behavioral observation probes and social validation measures served as the primary dependent variables. Results indicated that (a) treatment was effective for virtually all behaviors across all subjects, (b) improvements occurred for both training and generalization scenes, and (c) behavioral performance was maintained one month following the termination of treatment.     

EVALUATION OF A SOCIAL INTERACTION COACHING PROGRAM IN AN INTEGRATED DAY-CARE SETTING

We used a multiple baseline design across teachers (with a reversal phase for 1 teacher) to evaluate the direct and indirect effects of a structured coaching procedure on the teaching behaviors of 3 day-care teachers. Structured coaching preceding daily caregiver routines resulted in (a) substantial increases in adult delivery of behavioral support of social interaction during group activities with 2- and 4-year-old children and (b) marked collateral increases in positive interactions of socially withdrawn children. Long-term maintenance effects were demonstrated by both the teachers and target children, and social validity measures indicated that the teachers rated coaching very positively on several dimensions. The results are discussed in relation to in-service training of day-care staff, the concept of coaching as a setting event, and the dissemination of teaching technology related to social interaction of young children.     

DEVELOPMENT AND EVALUATION OF AN INFANT-CARE TRAINING PROGRAM WITH FIRST-TIME FATHERS

We evaluated the effectiveness of a multicomponent package in training infant-care skills to first-time fathers. After developing and socially validating a set of infant-care skills, we assessed the effects of training in a hospital-based program with expectant fathers (Experiment 1) and in a home-based program with fathers having varied degrees of experience with their infants (Experiment 2). In both experiments, a multiple probe design demonstrated that the training package was responsible for producing criterion performance by the expectant and first-time fathers. A 1-month generalization probe in Experiment 1 showed that the effects transferred across training conditions (training doll to human infant) and settings (hospital to home). An increase in the number of infant-stimulation activities performed by fathers was also observed in both experiments.     

PROGRAM EVALUATION RESEARCH: AN EXPERIMENTAL COST-EFFECTIVENESS ANALYSIS OF AN ARMED ROBBERY INTERVENTION PROGRAM

An armed robbery alarm system was implemented in 48 different stores in two separate geographical areas for 6 months and 12 months, respectively. The alarms were placed in the two separate areas at different times and all alarms were eventually removed. Thus, multiple baseline and reversal strategies were used to evaluate program impact. A device planted in a cash drawer was triggered whenever “bait” money was removed from the drawer sending an alarm signal directly to police cars and headquarters. Onscene apprehensions of armed robbers within target stores were greatly increased even though the armed robbery systems did not deter robbery incidents nor influence the court disposition of the cases. There was also no crime deterrence, crime displacement, or increased apprehensions in either the immediate neighborhoods of target stores or on a city-wide basis. The cost effectiveness of the program was calculated to be poor even though the program is being maintained because of the absence of an alternative robbery apprehension technology.     

THE RIGORS OF TRAINING EVALUATION: A DISCUSSION AND FIELD DEMONSTRATION1

Training evaluation is one of the most under-researched and neglected areas of industrial/organizational psychology. This article discusses the need for, and feasibility of, utilizing extended-control-group pretest designs in evaluation research; and reports a field application of such a design (the Solomon 4-Group) in an organizational training context. The effectiveness of a basic electricity training program for telephone installer-repairmen was measured, and the potential contaminating effects of pretesting were monitored. The results indicated that the training was potentially effective, but that pretest contaminations were present. To identify the pretest effects it was necessary to go beyond the Solomon model and consider complex interactions involving numerical aptitude level, pretesting and training. Pretest exposure depressed the posttest performance of trained subjects of medium and low numerical aptitude, while facilitating that of medium level untrained subjects. Pretest exposure had no effect on subjects scoring high in numerical aptitude in either the trained or untrained condition. Possible explanations for the moderating role of numerical aptitude are offered, and implications of the findings are discussed relative to applied organizational evaluation efforts and future research.     

AN EVALUATION OF ORIENTATION AND ASSESSMENT EFFECTIVENESS IN THE WORK INCENTIVE PROGRAM

The purpose of this study was to evaluate the effectiveness of a Work Incentive (WIN) Orientation and Assessment Program which was designed to (a) assess enrollee attitudes and abilities, and (b) provide enrollee pre-employment test orientation. A sample of 155 WIN enrollees was divided into two groups; the experimental group received two weeks of orientation and assessment. Results were obtained by using two criteria: First, the number of terminations prior to completing the EDP was examined for each group, and second, the total number of referrals to each of three major components was examined. Findings revealed a significantly lower number of terminations among the experimental group, but did not yield any difference in the number of referrals per component. It is suggested that further studies of this type include a more refined measure of enrollee success and satisfaction in various WIN components.     

BEYOND FORMAL EXPERIMENTAL DESIGN: TOWARDS AN EXPANDED VIEW OF THE TRAINING EVALUATION PROCESS

Textbook treatments of training evaluation typically equate evaluation with the measurement of change and focus on formal experimental design as the mechanism for controlling threats to the inference that the training intervention produced whatever change was observed. This paper notes that two separate questions may be of interest: How much change has occurred? and, Has a target performance level been reached? We show that the evaluation mechanisms needed to answer the two types of questions are markedly different, and discuss circumstances under which an evaluator's interests will focus on one, the other, or both of these questions. We then discuss alternatives to formal design as mechanisms for reducing various threats to validity, and discuss trade-offs between internal validity and statistical conclusion validity.     

THE BEHAVIOR OBSERVATION INSTRUMENT: A METHOD OF DIRECT OBSERVATION FOR PROGRAM EVALUATION1

The background and development of a multicategory direct observation system, the Behavior Observation Instrument (BOI), is described. This time-sampling procedure for recording the behavior of persons is demonstrated in several treatment settings and the results applied to issues of program evaluation. Elements that have prevented direct observation from being widely adopted, such as costs, manpower, and training requirements, are systematically analyzed. A basic psychometric analysis of the instrument is used to determine optimum frequency and duration of observation intervals as well as observer agreement. The results imply that direct observation methods, once assumed by some to belong to the special province of the single-subject design, can be used to assess the effects of programs on groups of psychiatric clients in an efficient and economic manner.     

TRAINING COUNSELING SKILLS: AN EXPERIMENTAL ANALYSIS AND SOCIAL VALIDATION

A community development perspective suggests the value of using local resources to solve local problems. Two low-income staff of a community service center served as nonprofessional counselors after receiving a training program consisting of written instructions, practice, and performance feedback. The effects of the 40-hour training program were positive for both counseling and problem-solving skills. There was also evidence of generalization of counseling performance across clients, problems, and time. In addition, expert judges' ratings of performance were obtained as a measure of social validity. These findings suggest that the training procedures are effective in enhancing the counseling and problem-solving skills of low-income nonprofessionals.     

ELIMINATION OF ECHOLALIC RESPONDING TO QUESTIONS THROUGH THE TRAINING OF A GENERALIZED VERBAL RESPONSE

Echolalia, the parroting of the speech of others, is a severe communication disorder frequently associated with childhood schizophrenia and mental retardation. Two echolalic children, one schizophrenic and one retarded, were treated in a multiple-baseline design across subjects. Each child was taught to make an appropriate, nonecholalic verbal response (i.e., “I don't know”) to a small set of previously echoed questions. After such training, this response generalized across a broad set of untrained questions that had formerly been echoed. The results obtained were the same irrespective of the specific experimenter who presented the questions. Further, each child discriminated appropriately between those questions that had previously been echoed and those that had not. Followup probes showed that treatment gains were maintained one month later. The procedure is economical, in that it produces a rapid and widespread cessation of echolalic responding.     

ACQUISITION OF EXPRESSIVE SIGNING BY AUTISTIC CHILDREN: AN EVALUATION OF THE RELATIVE EFFECTS OF SIMULTANEOUS COMMUNICATION AND SIGN-ALONE TRAINING

Two methods of training autistic children to use manual signs were compared. Two children, one mute and one capable of some verbal imitation, were taught to use signs as expressive labels for pictures of objects. Using an alternating treatments design, speed of sign acquisition was compared across two training conditions in which signs were presented either accompanied by, or without, the corresponding verbal label. In both conditions, the training procedure incorporated reinforcement, modeling, prompting, fading, and stimulus rotation. The efficacy of training in both treatment conditions was demonstrated by the use of a multiple baseline control across signs, but no clear differences in acquisition speed across conditions were apparent. Posttests conducted to assess stimulus control of signing, and learning of verbal labels when these were present in training, showed that the behavior of the imitative, but not the mute, child was controlled by the verbal stimuli. The implications of the results both for understanding deficits characteristic of autistic children and for developing appropriate language training procedures are discussed.     

SOME DIRECT AND GENERALIZED EFFECTS OF REPLACING AN AUTISTIC MAN'S ECHOLALIA WITH CORRECT RESPONSES TO QUESTIONS

We extended the use of operant procedures to decrease immediate echolalia and increase the appropriate responding to questions of a 21-year-old autistic man. Three experiments were conducted in which the overall plan was to (a) encourage the subject to remain quiet before, during, and after the presentation of questions and (b) teach him to use environmental cues (i.e., word cards or a model's responses) to increase the likelihood of responding correctly. Multiple baseline designs demonstrated that echolalia was rapidly replaced with correct stimulus-specific responses. In addition, there were a variety of generalized improvements in the subject's verbal responses to questions. The procedures and results are contrasted to previous research in an attempt to explain the encouraging findings.