EVALUATION OF AN INTERVIEWEE SKILLS TRAINING PROGRAM IN A NATURAL FIELD EXPERIMENT

The purpose of this study was to provide an evaluation of an interviewee skills training program in a field setting using a broad sample of interviewers, jobs, and candidates. The training program ( n = 158) derived its content from previous literature and used multiple instructional techniques. An experimental design was employed with a self-study (placebo control) group ( n = 140), random assignment, and many levels of evaluation criteria. Measures exhibited acceptable reliability, and statistical power was high for all analyses. Results indicated that class members responded positively to the program and demonstrated substantial learning. Interviewer evaluations of behavior failed to distinguish between experimental groups, however, and no differences were observed in job offers. Furthermore, no differences were observed between study participants and nonparticipants ( n = 174) in terms of interview behavior and job offers. This study illustrates the danger of relying solely on reaction or learning criteria in the evaluation of training programs. Several explanations are offered for the findings.     

AN OBJECTIVE EVALUATION OF A BEHAVIOR MODELING TRAINING PROGRAM

Past research on the effects of behavior modeling training has rarely focussed on actual changes in job performance. This paper describes a study in which results of a behavior modeling training program for sales representatives were evaluated in relation to effects on the sales performance of the participants. Sales associates selling large appliances, radios, and television sets for a large chain retailer in seven stores in one metropolitan area participated in a behavior modeling training program designed to improve their sales effectiveness. Before and after comparisons were made in their sales records with similar sales associates in seven matched stores who participated in other kinds of sales training during the same period. Sales representatives who received the behavior modeling training increased their sales by an average of 7% during the ensuing six-month period, while their counterparts in the control group stores showed a 3% decrease in average sales. The extra effort entailed in obtaining the needed sales records seemed to be worthwhile in providing convincing evidence of the value of the training.     

THE INFLUENCE OF TRAINEE ATTITUDES ON TRAINING EFFECTIVENESS: TEST OF A MODEL

Studies of the influence of trainee characteristics on training effectiveness have focused on the ability level necessary to learn program content. Motivational and environmental influences on training effectiveness have received little attention. The purpose of this study was to test an exploratory model describing the influence on trainee career and job attitudes on training outcomes (learning, behavior change, performance improvement). Results of the study suggest that job involvement and career planning are antecedents of learning and behavior change. Future research directions and practical implications of the results are discussed.     

Aligning job redesign with leadership training to improve supervisor support: a quasi-experimental study of the integration of HR practices

In this quasi-experimental study, we examine the alignment of a job redesign initiative with leadership training aimed at increasing supervisors’ opportunities for providing support to employees. In addition, we examine intervention-mediated effects on climate for innovation through increases in perceived supervisor support. To test the hypothesized process, we used employee ratings (N = 524) of perceived supervisor support and climate for innovation collected at three time points over 2 years in the home help services in seven Swedish municipalities. Results of latent growth curve analyses showed that employees in the intervention group had a stronger and positive slope of perceived supervisor support relative to the comparison group. Further, the growth trajectories of perceived supervisor support were positively associated with climate for innovation at the 24-month follow-up. The study contributes to the human resource management literature by showing that alignment of employment practices such as training with work practices such as job redesign may be a promising strategy for achieving positive outcomes at multiple levels in organizations.     

STATISTICAL POWER OF TRAINING EVALUATION DESIGNS

Sample size requirements needed to achieve various levels of statistical power using posttest-only, gain-score, and analysis of covariance designs in evaluating training interventions have been developed. Results are presented which indicate that the power to detect true effects differs according to the type of design, the correlation between the pre- and posttest, and the size of the effect due to the training program. We show that the type of design and correlations between the pre- and posttest complexly determine the power curve. Finally, an estimate of typical sample sizes used in training evaluation design has been determined and reviewed to determine the power of the various designs to detect true effects, given this sample-size specification. Recommendations for type of design are provided based on sample size and projected correlations between pre- and posttest scores.     

THE VALIDATION OF A MINICOURSE FOR TELEPHONE COMPANY SWITCHING TECHNICIANS1

The introduction of electronic switching equipment has changed the nature of the telephone company switching job. A lengthy and complex training program must be completed before an employee can perform the electronic switching job. Because of the high cost of this training a more elaborate, second-stage selection procedure was developed. The ESS Minicourse was designed to be a self-paced content valid sample of ESS training which would be suitable for use with job candidates without any previous telephone company experience. A criterion-related validity study was undertaken to provide further evidence of validity as well as data helpful in setting a cutting score. Results showed that a combination of time to complete the Minicourse and performance on the objective tests was predictive of time to complete self-paced training in electronic switching. Cross- validated estimates of validity were used to develop estimates of u'tility given different selection ratios.     

A QUANTITATIVE APPROACH TO EVALUATING TRAINING CURRICULUM CONTENT SAMPLING ADEQUACY

A technique was developed which provided a quantitative index of the fit between training curriculum content and job task performance requirements. The procedure also generates a listing of tasks which receive undue emphasis in the training curriculum, those which are not being trained, and those which instructors intend to train, but which course graduates report being unable to perform. The procedure is illustrated for three training programs in the U.S. Coast Guard Class 'A' schools.     

BEHAVIORAL TRAINING FOR SIBLINGS OF AUTISTIC CHILDREN

The present study was conducted to investigate the effectiveness of a program designed to teach behavior modification procedures to normal siblings of autistic children. Three sibling pairs participated in a multiple-baseline analysis of the effects of training the normal siblings to use behavior modification procedures to teach their autistic brother or sister a variety of learning tasks. Results indicated that the siblings learned to use the behavioral procedures at a high level of proficiency, they used the procedures in a generalization setting, and there were observed improvements in the behavior of the autistic children. In addition, a social validation assessment of the normal siblings' statements about their autistic sibling indicated a decrease in negative statements and an increase in positive statements after training. These results are discussed in terms of the potential for incorporating siblings into the treatment plan in intervention programs with autistic children.     

TRAINING PEERS TO TUTOR: EVALUATION OF A TUTOR TRAINING PACKAGE FOR PRIMARY LEARNING DISABLED STUDENTS

Peer tutoring—academic instruction given by a student to a same-aged classmate—can be a cost-effective way of providing needed instruction in the typical classroom where one teacher must deal with many students of varying ability levels. This study evaluated an instructional package as a training method for increasing the use of selected tutoring behaviors by peer tutors. Nine male students, aged 7 to 9 years, from a learning disabilities resource room served as subjects; four subjects served as tutors, four as students, and one as a generalization student who was assigned to all tutors during generalization testing. Subjects were assigned to tutor-student pairs based on their performances on a pretest of the 220 Dolch Basic Sight Words, with the subjects scoring highest on the pretest serving as tutors. Each tutor worked with his assigned student throughout the study, except for pre- and posttest sessions with the generalization student. During daily sessions, tutors trained their students to recognize sight words using 10 flash cards, each with 1 of the Dolch words printed on it. Training was conducted with all four tutors in a group and was evaluated using a multiple baseline design across three classes of tutoring behaviors: Preparation Behaviors, Prompting Behaviors, and Praising Behaviors. Training consisted of: (a) verbal instructions on how to perform the tutoring behaviors; (b) charts illustrating the appropriate tutoring behaviors with stick figures; (c) role playing with each tutor with the experimenter playing the part of the student and giving feedback and verbal reinforcement to the tutor for performance of the appropriate tutoring behaviors; and (d) verbal testing of each tutor by asking him to state orally the behaviors he was to perform. Training sessions took approximately 30 minutes for each of the 3 classes of tutoring behaviors. Results showed an increased use of tutoring behaviors by tutors as a function of the instructional package. Increases in tutoring behaviors were also observed during postintervention generalization testing of tutors instructing their regular student in new subject matter (math). Data taken from pre-, mid-, and postintervention testing of the 220 Dolch Words showed an increase of identifiable sight words by both students and tutors after completing the peer tutoring program. This study demonstrates that a simple tutor training package can be employed to teach young, special education students to serve as effective tutors for their peers. In addition, the academic gains evidenced by the subjects lend positive support to the effectiveness of peer tutoring programs and specifically to the research of Dineen, Clark, and Risley (Journal of Applied Behavior Analysis, 1977, 10 , 231–238) which demonstrated the educational benefits of tutoring incurred by the tutor as well as by the student.     

DEVELOPMENT AND EVALUATION OF AN INFANT-CARE TRAINING PROGRAM WITH FIRST-TIME FATHERS

We evaluated the effectiveness of a multicomponent package in training infant-care skills to first-time fathers. After developing and socially validating a set of infant-care skills, we assessed the effects of training in a hospital-based program with expectant fathers (Experiment 1) and in a home-based program with fathers having varied degrees of experience with their infants (Experiment 2). In both experiments, a multiple probe design demonstrated that the training package was responsible for producing criterion performance by the expectant and first-time fathers. A 1-month generalization probe in Experiment 1 showed that the effects transferred across training conditions (training doll to human infant) and settings (hospital to home). An increase in the number of infant-stimulation activities performed by fathers was also observed in both experiments.     

THE MINIATURE JOB TRAINING AND EVALUATION APPROACH: ADDITIONAL FINDINGS

Miniature job training and evaluation situations were developed and administered to 1034 "low aptitude" Navy recruits. Checklist criterion data describing the on-the-job performance of the sample were collected after the recruits were on their fleet assignments: (1) nine months, and (2) 18 months. The results confirmed prior findings relative to the predictive validity of the miniature job training and evaluation approach and supported contentions favoring the power of the concept over a paper-and-pencil testing approach.     

OPTIMIZING KNOWLEDGE TESTS FOR INFERRING LEARNING ACQUISITION LEVELS IN SINGLE GROUP TRAINING EVALUATION DESIGNS: THE INTERNAL REFERENCING STRATEGY

Training evaluation is a process designed to measure the degree of trainee changes (on learning as well as on other dimensions) that training programs cause. Whereas many experimental and correlational designs permitting such inferences exist, they are rarely used, principally for practical reasons. This paper evaluates an operationally simple procedure for estimating effectiveness of training in improving trainee knowledge–the Internal Referencing Strategy (IRS). It identifies and tests the implicit training evaluation notion that training-relevant content should show more change (pre-post) than training-irrelevant content. An empirical evaluation study conducted with 66 managers enrolled in a training course compares the inferences produced by the IRS versus a more traditional experimental evaluation. The results indicate that the IRS approach may permit inferences that mirror those obtained by the more complex design. The paper closes with a discussion of the limits and possible uses of this design.     

Factors that Precipitate and Mitigate Crises in Ministry

Using a modified Delphi technique, this study reports mixed-method data for factors identified as leading to crises in ministry. Data were collected from a random stratified sample of Uniting Church clergy based in southern Australia. Three phases (Phase 1?N?=?85, Phase 2?N?=?95, Phase 3?N?=?75) identified precipitants to crises in ministry and strategies to mitigate these crises. Strategies included a need for ongoing mandatory professional development, professional supervision, peer support, and formal ethics training. Results are discussed in the light of job demands, job control and job support, and the self-determination needs of competence, relatedness, and autonomy.     

Employee and Supervisor Ratings of Motivation: Main Effects and Discrepancies Associated with Job Satisfaction and Adjustment in a Factory Setting1

Research and theory on employee job satisfaction and well-being has increasingly concentrated on both intrinsic and extrinsic motivational factors. According to self-determination theory (Deci & Ryan, 1985). autonomy, relatedness, and competence are three intrinsic psychological needs that, if fulfilled in the workplace, will lead to greater satisfaction, performance, and general well-being. This study examines employee and supervisor perceptions of the employee's autonomy, competence, and relatedness in the workplace, as well as the degree and direction of discrepancies between employee and supervisor reports. Both employee and supervisor ratings of intrinsic motivational factors were significantly related to work satisfaction, psychological health, and self-esteem, after controlling for the extrinsic factors of pay and job status. Results of discrepancy analyses were somewhat supportive of overrating being associated with greater well-being and job satisfaction. Discussion of the results ties this study to relevant research from a self-determination perspective and to the growing literature on discrepancies and self-perception.     

A META-ANALYSIS OF THE VALIDITY OF METHODS FOR RATING TRAINING AND EXPERIENCE IN PERSONNEL SELECTION

This paper reviews the validity evidence for systematic methods used to evaluate training and experience (T&E) ratings in personnel selection. Meta-analytic summaries of the data indicate that validity varies with the type of T&E evaluation procedure used. The lllinois job element and behavioral consistency methods each demonstrated useful levels of validity (.20 and .45, respectively) with small corrected standard deviations, thus supporting validity generalization. Both the point and task methods yielded low mean validities (.11 and .15, respectively) with larger variability. The authors hypothesized that both the point and task methods were affected by a job experience moderator. Partial support for this hypothesis was found. Moderator analyses suggested that the point method was most valid when the applicant pool had low mean levels of job experience and was least valid with an experienced applicant pool. Additional research is desirable on all T&E methods to decrease the potential impact of second-order sampling error in the meta-analytic results. Further research is also needed to explicate the constructs measured by T&E evaluations.     

SOCIAL EXCHANGE AND ORGANIZATIONAL COMMITMENT: DECISION-MAKING TRAINING FOR JOB CHOICE AS AN ALTERNATIVE TO THE REALISTIC JOB PREVIEW

This field experiment investigated the effects of exchange-inducing treatments on pre- and postentry commitment of military recruits. Behavioral (volunteering for combat service and turnover), intentional (willingness to commit to combat service), and attitudinal (commitment, satisfaction, perceived fairness, and perceived choice variety) outcomes are examined. Two exchange-inducing experimental groups, one receiving realistic job preview and another receiving decisionmaking training, were compared to 3 control groups. Results indicated that preentry commitment was significantly higher among participants in the exchange-inducing conditions. However, the effect of decisionmaking training lasted longer than the effect of realistic job preview.     

Intrinsic need satisfaction and the job attitudes of volunteers versus employees working in a charitable volunteer organization

This research examines how intrinsic need satisfaction during volunteer work relates to the job attitudes of volunteers ( Deci & Ryan, 2000 ; Ryan & Deci, 2000 ). We distinguish between autonomy, competence, and relatedness needs, and predict these to have distinct effects on volunteers' job satisfaction and intent to remain with the volunteer organization. We then compare volunteer workers to a matched sample of paid employees (performing identical tasks within the same organization) in the way in which they derive their job satisfaction and intent to remain from satisfaction of these three types of needs. As predicted, path analysis (N=105) shows that satisfaction of autonomy needs and relatedness needs are positively related to the intent to remain a volunteer with the volunteer organization, and this relation is mediated by satisfaction with the volunteer job. The matched samples comparison further reveals that whereas the job satisfaction and intent to remain of paid employees is most clearly predicted by satisfaction of autonomy needs, satisfaction of relatedness needs is the primary predictor of job satisfaction and intent to remain among volunteers. Theoretical and practical implications, as well as avenues for further research, are discussed.