PROGRAM EVALUATION

The following article is a first in a series to be offered by the Professional Development Committee and your Journal. Each article will feature some resource materials which might be used by the individual counselor or school. The materials are offered as possible means for the continued growth and development of counseling and guidance within our parochial schools. Your reactions to the basic idea, format and materials are eagerly solicited.     

A FIVE-YEAR STUDY OF A POSITIVE INCENTIVE ABSENCE CONTROL PROGRAM

This study reports the results of a positive incentive program designed to reduce absenteeism at a hospital. Absence data for a treatment group ( N = 164) and a comparison group ( N = 136) were collected for one year prior to the incentive program, the three years the plan was operational, and the year after the program was discontinued. In the treatment program, absence from the treatment group decreased significantly during the first and third years of the program; effects during the second year of the program were marginally significant. No changes occurred in the comparison group's absence level. A utility analysis revealed that the incentive program produced an 11.7% return on investment.     

A TOKEN REINFORCEMENT PROGRAM IN A PUBLIC JUNIOR-HIGH SCHOOL1

In recent years, the technology of contingency management has been shown to be of increasing value in regular classrooms and public-school systems with both groups and individual pupils (Ayllon and Roberts, Journal of Applied Behavior Analysis, 1974, 7 , 71–76; Glynn and Thomas, Journal of Applied Behavior Analysis, 1974, 7 , 299–306; Lovitt and Curtiss, Journal of Applied Behavior Analysis, 1969, 2 , 49–53; Lovitt and Smith, Exceptional Children, 1974, 40 , 357–358; Medland and Stachnik, Journal of Applied Behavior Analysis, 1972, 5 , 45–51). In addition, established procedures are being systematically replicated across grade levels and differing subject-matter areas. A series of studies initiated by O'Leary and Becker (Exceptional Children, 1967, 33 , 637–642) form the basis for the present investigation. The token reinforcement program described by O'Leary and Becker (Exceptional Children, 1967, 33 , 637–642) was managed by the teacher of an adjustment class and involved 9-yr-old children described as emotionally disturbed. An elaborate replication of the 1967 O'Leary and Becker study conducted by O'Leary, Becker, Evans, and Saudargas (Journal of Applied Behavior Analysis, 1969, 2 , 3–31) with a grade-two class introduced several variables to examine their separate effects. The authors specified their treatment levels as baseline, classroom rules, educational structure, teacher praise and ignore, token I, withdrawal, token II, and follow-up. The present research modified the general design of O'Leary et al., (Journal of Applied Behavior Analysis, 1969, 2 , 3–31) to study how to maintain treatment effects. In the modification, rules were eliminated as a treatment variable because they are frequently associated with aversive practices in the school system, and it was deemed desirable to have mainly a positive orientation. A second difference was that the present subjects were grade-nine pupils functioning in the regular public-school system. The six students were older (average age = 16.2) and well behind their peers in achievement. They were considered behavior problems and as potential dropouts by teachers and counsellors. They were not considered to be emotionally disturbed. Finally, procedures designed to maintain behavior change generated by the token system were added. The operant level of unacceptable classroom behavior was obtained for six students receiving an individualized program of instruction in mathematics and science in a nonacademic grade-nine class in a public junior secondary school. Initially, two conditions (educational structure and praising appropriate behavior while ignoring inappropriate behavior) were introduced successively. Both procedures reduced inappropriate behavior slightly. When a token system, using backup reinforcers readily available in the school, was introduced in conjunction with the initial two conditions, inappropriate responses declined dramatically in all subjects. Withdrawal of the token program for a three-week period, leaving educational structure and praising and ignoring in effect, increased inappropriate behavior in five of the six subjects. The token program was then re-introduced in conjunction with contingency contracts. The result was a decline of inappropriate behavior below the mean of the first token condition for all subjects. Tokens were thinned and finally removed toward the end of this condition, leaving teacher praise and attention and the contract system in effect. Data obtained during a four-week followup indicated that the low level of inappropriate behavior was maintained in all subjects. This extension of the O'Leary et al., (Journal of Applied Behavior Analysis, 1969, 2 , 3–31) program, designed and implemented by the regular teacher, demonstrates that these procedures may be highly effective within the constraints found in an ordinary classroom in the junior secondary school.     

AN OBJECTIVE EVALUATION OF A BEHAVIOR MODELING TRAINING PROGRAM

Past research on the effects of behavior modeling training has rarely focussed on actual changes in job performance. This paper describes a study in which results of a behavior modeling training program for sales representatives were evaluated in relation to effects on the sales performance of the participants. Sales associates selling large appliances, radios, and television sets for a large chain retailer in seven stores in one metropolitan area participated in a behavior modeling training program designed to improve their sales effectiveness. Before and after comparisons were made in their sales records with similar sales associates in seven matched stores who participated in other kinds of sales training during the same period. Sales representatives who received the behavior modeling training increased their sales by an average of 7% during the ensuing six-month period, while their counterparts in the control group stores showed a 3% decrease in average sales. The extra effort entailed in obtaining the needed sales records seemed to be worthwhile in providing convincing evidence of the value of the training.     

SUPERVISING PARAPROFESSIONAL TUTORS IN A REMEDIAL READING PROGRAM1

Applied researchers increasingly report the use of paraprofessionals to implement key program components. However, despite such apparent advantages as increased availability and lower salaries, problems in maintaining acceptable levels of on-the-job performance in these workers have been reported. This study assessed the effects of a supervisory package on the work behavior of five paraprofessional tutors in a remedial reading program. The package consisted of written handouts and instructions, tests of tutor understanding, a video tape, mention of tutor performance by supervisors, and publicly posted feedback on work performance. One randomly chosen tutor received feedback each day on (1) his degree of completeness in tutoring one student's answers to comprehension-check questions, (2) his accuracy in tabulating that student's data sheet, and (3) his promptness in beginning the first student's tutorial session. The supervisory package produced marked improvement in completeness, some improvement in accuracy, but no improvement in promptness. Application of the supervisory package seemed practical, as an average of nine daily tutorial sessions (approximately 270 min of tutoring) required a total daily average of only 28 min of supervision. It was concluded that completeness performance by nondegreed, paraprofessional tutors was closely related to the extent to which they were supervised. Despite the fact that no improvement was observed in tutor promptness, and only partial improvement was observed in tutor accuracy, the improvement to near-perfect levels in tutor completeness suggests that further research is warranted to develop supervisory packages that might ensure the reliable and efficient use of paraprofessional workers.     

CO-WORKER SOCIAL SUPPORT IN A WORKSITE SMOKING CONTROL PROGRAM

We evaluated the effects of adding a social support component to a worksite controlled smoking treatment program. Twenty-four participants were randomly assigned to either a controlled smoking or a controlled smoking plus partner support condition. Within a multiple baseline across behaviors design, smokers in both conditions made efforts to achieve sequential 50% reductions in: (a) nicotine content of brand smoked, (b) number of cigarettes smoked per day, and (c) percentage of each cigarette smoked. Self-monitoring records, laboratory analyses of spent cigarette butts, and carbon monoxide determinations indicated that both conditions were effective in producing significant reductions in each of the three target behaviors and in carbon monoxide levels. All participants who quit smoking during the program maintained their abstinence at a 6-month follow-up, and those who did not quit were smoking less at follow-up than they had at pretest on all dependent variables. However, few differences were observed between controlled smoking and controlled smoking plus partner support conditions either during treatment or at the 6-month follow-up. Results are discussed with regard to previous worksite studies, future directions for research on social support, and variables that may have mediated treatment outcome.     

THE RELATIONSHIP BETWEEN FORMAL MENTORING PROGRAM CHARACTERISTICS AND PERCEIVED PROGRAM EFFECTIVENESS

Formal mentoring programs continue to gain popularity within organizations despite limited empirical research regarding how these programs should be designed to achieve maximum effectiveness. This study examined design features of formal mentoring programs and perceived program effectiveness from both mentor and protégé perspectives. Mentor commitment and program understanding were examined as mediators. Substantial support for the proposed path analytic model was found. These results help begin to bridge the gap between science and practice concerning the design of formal mentoring programs.     

ACADEMIC LEVEL OF POVERTY PROGRAM ENROLLEES: A NEW LOOK

The Adult Basic Learning Examination was administered to 393 enrollees in a Concentrated Employment Program. On the basis of this investigation it was found that trainees obtained higher scores in the language skills than in computational areas. The suitability of the ABLE as a screening instrument for poverty programs is discussed.     

A PERFORMANCE IMPROVEMENT PROGRAM FOR AN INTERNATIONAL-LEVEL TRACK AND FIELD ATHLETE

This study outlined the development, implementation, and evaluation of an intervention strategy that was designed to improve the technical skill and performance of an international-level pole vaulter. Intervention, in the form of a prompting and shaping procedure, consisted of breaking a photoelectric beam with the hands at the moment of take-off. The height of the beam was gradually increased until the vaulter reached maximum arm extension at take-off. Increase in arm extension was matched by an increase in bar height clearance.     

EFFICACY AND MAINTENANCE OF AN EDUCATION PROGRAM FOR A CONSUMER COOPERATIVE

We examined the effects of contingency management on participation in and maintenance of an education program by new members of a student housing cooperative. With credit and fine contingencies in place, the percentage of participants completing study guides was five times higher than without the contingencies. Members continued to implement the program for 9 years without researcher involvement.     

PROGRAM EVALUATION RESEARCH: AN EXPERIMENTAL COST-EFFECTIVENESS ANALYSIS OF AN ARMED ROBBERY INTERVENTION PROGRAM

An armed robbery alarm system was implemented in 48 different stores in two separate geographical areas for 6 months and 12 months, respectively. The alarms were placed in the two separate areas at different times and all alarms were eventually removed. Thus, multiple baseline and reversal strategies were used to evaluate program impact. A device planted in a cash drawer was triggered whenever “bait” money was removed from the drawer sending an alarm signal directly to police cars and headquarters. Onscene apprehensions of armed robbers within target stores were greatly increased even though the armed robbery systems did not deter robbery incidents nor influence the court disposition of the cases. There was also no crime deterrence, crime displacement, or increased apprehensions in either the immediate neighborhoods of target stores or on a city-wide basis. The cost effectiveness of the program was calculated to be poor even though the program is being maintained because of the absence of an alternative robbery apprehension technology.