A SIMPLIFIED TIME-SERIES ANALYSIS FOR EVALUATING TREATMENT INTERVENTIONS

Time-series analysis procedures for analyzing behavioral data are receiving increasing support. However, several authorities strongly recommend using at least 50–100 points per experimental phase. A complex mathematical model must then be empirically developed using computer programs to extract serial dependency from the data before the effects of treatment interventions can be evaluated. The present discussion provides a simple method of evaluating intervention effects that can be used with as few as 8 points per experimental phase. The calculations are easy enough to do by hand.     

DEVELOPMENT ENGAGEMENT WITHIN AND FOLLOWING DEVELOPMENTAL ASSESSMENT CENTERS: CONSIDERING FEEDBACK FAVORABILITY AND SELF–ASSESSOR AGREEMENT

This study sought to understand employees' level of behavioral engagement in response to feedback received in developmental assessment center (DAC) programs. Hypotheses were drawn from theories of self‐enhancement and self‐consistency and from findings in the multisource feedback and assessment center literatures regarding recipients' perceptions of feedback. Data were gathered from 172 U.S. middle managers participating in a DAC program. Results suggested that more favorable feedback was related to higher behavioral engagement. When discrepancies between self‐ and assessor ratings were examined, overraters (participants whose overall self‐ratings were higher than their assessor ratings) tended to show less engagement in the program compared to underraters. However, pattern agreement on the participant's dimension profile did not significantly correlate with behavioral engagement. Based on these findings, avenues for future research are presented and practical implications are discussed.     

CONSTRUCTING PARALLEL SIMULATION EXERCISES FOR ASSESSMENT CENTERS AND OTHER FORMS OF BEHAVIORAL ASSESSMENT

Assessment centers rely on multiple, carefully constructed behavioral simulation exercises to measure individuals on multiple performance dimensions. Although methods for establishing parallelism among alternate forms of paper-and-pencil tests have been well researched (i.e., to equate tests on difficulty such that the scores can be compared), little research has considered the why and how of parallel simulation exercises. This paper extends established procedures for constructing parallel test forms to dimension-based behavioral simulations. We discuss reasons for establishing comparable, alternate simulation forms and discuss the issues raised when applying traditional procedures to simulation exercises. After proposing a set of guidelines for establishing alternate forms among simulations, we apply these guidelines to simulations used in an operational assessment center.     

ASSESSMENT CENTERS AND CONTENT VALIDITY: SOME NEGLECTED ISSUES

Literature on the use of content validity to establish the job-relatedness of assessment centers is reviewed. Most approaches to content validity are shown to focus on job-analysis-based exercise construction and dimension selection as the bases for determining the content validity of an assessment center. The article attempts to show that content validity requires more than the careful construction of stimulus materials (i.e., exercises); how these stimulus materials are presented to candidates and how responses to these stimuli are evaluated are also critical considerations in making judgments about content validity. Variability in the way these issues are dealt with is documented.     

EXERCISES AND DIMENSIONS ARE THE CURRENCY OF ASSESSMENT CENTERS

A novel assessment center (AC) structure that models broad dimension factors, exercise factors, and a general performance factor is proposed and supported in 4 independent samples of AC ratings. Consistent with prior research, the variance attributable to dimension and exercise factors varied widely across ACs. To investigate the construct validity of these empirically supported components of AC ratings, the nomological network of broad dimensions, exercises, and general performance was examined. Results supported the criterion‐related validity of broad dimensions and exercises as predictors of effectiveness and success criteria as well as the incremental validity of broad dimensions beyond exercises and general performance. Finally, the relationships between individual differences and AC factors supported the construct validity of broad dimension factors and provide initial insight as to the meaning of exercise specific variance and general AC performance.     

HYPOTHESIS-BASED INTERVENTIONS FOR TANTRUM BEHAVIORS OF PERSONS WITH DEVELOPMENTAL DISABILITIES IN SCHOOL SETTINGS

We conducted a functional assessment of problem behaviors of 2 students with developmental disabilities in their classroom environments. Results of the assessments showed that although there were more tantrums in demand than in no-demand conditions, the function of the behavior was to gain attention (positive reinforcement) rather than to avoid or escape demands (negative reinforcement); demand conditions apparently served a discriminative function for the availability of attention. Therefore, intervention was based on the positive reinforcement hypothesis, resulting in a substantial reduction of tantrums for both subjects.     

REINFORCER ASSESSMENT FOR CHILDREN WITH DEVELOPMENTAL DISABILITIES AND VISUAL IMPAIRMENTS

We assessed the applicability of two previously developed reinforcer assessment procedures to children with developmental disabilities and visual impairments. Greater differentiation between stimuli was observed with a choice procedure than with a preference procedure. Measurement of compliance and rate of responding in adaptive skill training confirmed that the choice procedure accurately identified reinforcing stimuli. The preference procedure produced false positive predictions of reinforcer efficacy.     

ASSESSMENT CENTERS AND SUBORDINATE APPRAISALS OF MANAGERS: A SEVEN-YEAR EXAMINATION OF PREDICTIVE VALIDITY

Assessment center ratings of 52 state law enforcement agency managers predicted both a contaminated promotions criterion and two uncontaminated ratings criteria, but they did significantly better in predicting the former. Criteria data were gathered two, four, and seven years after the assessment center, and a modest upward trend in validity coefficients was observed. Ratings by subordinates demonstrated promise as a near term predictor, outperforming the assessment center on uncontaminated criteria in the two- and four-year time frames. Last, regression analyses using Year 7 criteria revealed unique predictive power for both ratings by subordinates and assessment centers above and beyond that provided by supervisor ratings.
An earlier version of this paper was presented at the Academy of Management meeting, Anaheim, California, August 7-10, 1988.     

IMPLICATIONS OF TRAIT-ACTIVATION THEORY FOR EVALUATING THE CONSTRUCT VALIDITY OF ASSESSMENT CENTER RATINGS

Assessment centers have often been criticized for lacking evidence supporting the construct validity of dimension ratings. This study examines whether the poor convergence of assessment center ratings is a result of correlating ratings from exercises that differ in the extent that behavior relevant to personality traits can be observed. Using data from a promotional assessment center for law enforcement officers ( n = 79), the convergence of assessment center ratings was evaluated within the context of the five factor model by comparing the average within-dimension correlation of ratings from exercises that allowed for more opportunity to observe trait-relevant behavior to the average of those involving exercises where there was less opportunity. For each personality trait, ratings from exercises judged by experts to be high in trait-activation potential displayed stronger convergence (mean r = .30) than did ratings from exercises that were low in activation potential for that trait (mean r = .15). Implications for evaluating the construct validity of assessment centers are discussed along with future directions for classifying exercises based on situational similarity.     

EVALUATING STRATEGIES TO IMPROVE CAREPROVIDER PERFORMANCE ON HEALTH AND DEVELOPMENTAL TASKS IN AN INFANT CARE FACILITY

Responding to administrative staff and parental concerns, using modified reversal and withdrawal designs, two experiments evaluated a staff-managed feedback system to improve the hygiene and developmental skills of children in an infant/toddler center. Experiment 1 examined feedback designed to increase staff performance in checking and changing diapers, and recording those changes. A chart plus supervisory feedback produced increases in and maintenance of staff performance. Experiment 2 compared an existing staff management system with a “playchart” plus feedback in increasing careprovider-infant stimulation. The data (with follow-up on a new staff) supported the use of the new feedback system. Questionnaire data further supported the utility of the playchart system.     

ASSESSMENT OF STEREOTYPIC AND SELF-INJURIOUS BEHAVIOR AS ADJUNCTIVE RESPONSES

Certain responses of both humans and nonhumans appear to be maintained indirectly by intermittent reinforcement schedules and have been referred to collectively as adjunctive behavior. Although basic research has examined adjunctive behavior extensively, relatively few studies have been conducted with humans, particularly those with developmental disabilities who often engage in frequent and varied stereotypic behavior. This study assessed possible adjunctive characteristics of self-injurious and stereotypic behaviors using a multielement design containing two types of control conditions. Four subjects who engaged in both self-injurious behavior and stereotypy participated after variables maintaining their self-injury were identified via functional analyses. Each day, subjects were exposed to three 15-min sessions in random order: (a) noncontingent presentation of food on a fixed-time schedule (e.g., FT 30 s), (b) a massed-reinforcement (food) control, and (c) a no-reinforcement control. A variety of fixed-time schedules were examined during different experimental phases. Results of this preliminary study suggested that self-injury was not induced by intermittent reinforcement schedules, whereas the stereotypic behavior of some individuals showed characteristics of adjunctive behavior. The importance of research on adjunctive behavior and suggestions for future studies are discussed.     

ASSESSMENT OF MOTION AS INFLUENCED BY STRUCTURE OF BACKGROUND

G ottsdanker , R. Assessment of motion as influenced by structure of background. Scand. J. Psychol ., 1962, 3 , 122–128.—Detection of target acceleration was improved by having a near landmark in addition to the surrounding frame. Adding landmarks, even within the course of motion, brought no further gain. Evidently, rate is not assessed by estimating time taken to cover known distances. Best performance was found when fine background marks were near enough together to form a texture, but not if it was perfectly uniform. It is suggested that the advantage provided both by the near point and the texture is the indirect one of stabilizing space. This effect is conceived of as graded, reaching its ultimate value in a textured surface.     

A NOTE ON EVALUATING BEHAVIOR MODIFICATION AND BEHAVIOR MODELING AS INDUSTRIAL TRAINING TECHNIQUES

The training literature (1967-1976) was searched for reports of scientific evaluations of behavior modification and behavior modeling used in industrial training. No reported scientific evaluations of behavior modification were found. Four reports (1976) of scientific evaluation of behavior modeling used in training managers were found. The research designs used in these studies were analyzed for possible threats to internal validity. Enough threats to internal validity were discovered in the designs used to question the reported results of behavior modeling training of managers.     

USE OF AN ASSESSMENT CENTER AS AN AID IN MANAGEMENT SELECTION

An application of assessment center procedures is described for the selection of first level managers from the craft ranks. Information is reported on how the assessment data are used in promotion decisions. An evaluation study was conducted that indicates that the assessment center program has a positive effect on performance of managers at the first level and the pool of potential for higher levels of management. The study results also suggest ways in which assessment data can be used more effectively in promotion decisions.     

DIMENSIONS AND CORRELATES OF THE PERSONNEL AUDIT AS AN ORGANIZATIONAL ASSESSMENT TOOL

This study analyzes the measurement properties of the personnel audit as an organizational assessment tool and the extent to which audit results relate to important organizational criteria. The findings reported here indicate that nine dimensions can effectively summarize 37 audit measures. The composite factor scores for these dimensions predict executive perceptions of "overall organizational performance" and "overall contribution of Personnel Function," employee "overall satisfaction with personnel services," and to a lesser extent, "profitability per capita." The personnel budget allotted per capita is in turn related to executive perceptions of Personnel's contributions.     

THE SPLIT LEVEL PROBLEM AS RELATED TO THE TASK OF EVALUATING ITEMS IN THE CLINICAL AREA

H olley , J. W. & H ansson , K. The split level problem as related to the task of evaluating items in the clinical area. Scand.J. Psychol ., 1969, 10, 124–128—It is pointed out that correlational values can be distorted in item evaluation when the proportions of persons in the criterion groups are variant. The use of a 'K ratio' is suggested as a possible solution to be applied under restrictive conditions.