Validation of a role-play measure of children's social skills

A videotape-administered role-play test of children's social skills was developed and its psychometric properties tested. Performance criteria for the test were derived from popular children's ratings of the effectiveness of different role-play responses. The test was administered to 157 fourth- and fifth-grade boys and girls who had been classified as popular, average, neglected, or rejected, on the basis of sociometric testing. The test evidenced good interrater, test-retest, and internal consistency reliabilities. Children's role-play performance correlated significantly with teacher ratings of social competence and with peer-liking ratings. Neglected children performed more poorly on the role-play test than popular children. When differences in intelligence among social status groups were statistically controlled, social status groups did not differ on the role-play test. Results of the discriminant analyses support the conclusion that teacher ratings are better than role-play tests for identifying rejected children, whereas role-play tests and measures of intelligence appear more accurate than teacher ratings for identifying neglected children.     

An investigation into the validity of role play as a procedure for counsellor skill assessment

The aim of this study was to determine the validity of role play as a method for assessing counsellor skill. Video recordings of twelve counsellors' performance in both real and simulated counselling interactions were analysed. Skill assessment included molar ratings of eight skill areas and molecular measures of 28 behaviours. A comparison of counsellor performance between conditions indicated that there was little difference between the real and role-play situations. It is concluded that the modified extended role-play format employed in the present study might be a valid technique for assessment of counselling skills.     

The relationship of role-play assessment of children's social skills to multiple measures of social competence

The present investigation evaluated the relationship of role-play performance of children's social skills with multiple measures of social competence. Thirty-eight psychiatric inpatient children (ages 7–13 yrs) completed measures to assess role-play performance, knowledge of social skills, self-efficacy and self-reported social behavior in diverse situations. Overt social interaction was directly but unobtrusively assessed in the hospital as well. Multiple measures of social skills were also completed by hospital staff, parents and teachers. The results indicated that role-play performance correlated significantly with child knowledge and self-efficacy but not with overt social behavior or self-reported social behavior in diverse situations. Role-play performance generally did not correlate with measures completed by staff, parents and teachers. Yet measures completed by these informants did converge and also were useful in delineating diagnostic groups of children whose social behavior would be expected to be problematic. The implications of the results for the use of role-play tests to evaluate social skills and to discriminate clinical populations are discussed.     

Recurrent problems in the behavioral assessment of social skill

Behavioral strategies for assessing social skill have been subjected to extensive analysis and criticism in the past few years. Many problems with earlier strategies have been corrected, yet the literature continues to be marked by a number of serious errors and invalid procedures. The purpose of this paper is to identify some of these recurrent difficulties, and recommend alternatives where possible. The discussion covers three general topics: the measures employed, the assessment format and conceptual issues which bear on assessment. Specific issues addressed include the respective value of molecular and molar ratings, the extent to which subjects actually engage in role playing, the utility of single prompt role-play tests, the selection and form of role-play scenarios and the role of skill deficits in interpersonal dysfunction. Some of the major assumptions and persistent dilemmas in the existing literature are examined in the light of questionable and often invalid assessment procedures.     

Comprehension of nonverbal communication: A reexamination of the social competencies of learning-disabled children

This study was designed to determine whether learning-disabled (LD) children differed from nondisabled (NLD) children in their ability to comprehend nonverbal communication when potential attentional differences between the groups were controlled. In addition, the relationship between nonverbal comprehension and social competence was assessed. Thirty LD and 30 NLD boys between 9 and 12 years of age were administered a short form of the Profile of Nonverbal Sensitivity (PONS) to assess nonverbal comprehension; social competence measures included teachers' ratings of aggressive and withdrawn behaviors using the Behavior Problem Checklist, and blind judges' ratings of performance on a role-play of friendshipmaking skills. Under attention-incentive conditions, no performance differences between LD and NLD children were found on the PONS; however, LD children were judged to be more withdrawn and less socially skilled. While PONS scores were not related to other social competence measures, hey were associated with academic achievement and IQ. Results emphasize the importance of considering the presence of attentional problems in LD children that may interfere with an accurate assessment of their skills.     

SOCIAL SKILLS TRAINING WITH CHILDREN: PROCEED WITH CAUTION

Three learning disabled children, selected on the basis of peer sociometric ratings and teacher referral, received social skills training. A group training procedure consisting of coaching, modeling, behavior rehearsal, and feedback was used to teach children the target behaviors of eye contact and appropriate verbal responses. A multiple baseline analysis across target behaviors was used to demonstrate treatment effectiveness on role-play scenes trained during treatment sessions. Duration of speech was measured as an untrained, corollary measure. The following measures were also obtained during baseline, posttreatment, and 1-mo follow-up for the experimental subjects and three control subjects: (a) performance on role-play scenes not trained during treatment sessions; (b) behavioral observations in a free play setting, and (c) sociometric ratings. In addition, the trained and untrained role-play scenes were administered by novel experimenters following treatment. The results indicated that socially unskilled, learning disabled children can be taught to respond appropriately to role-play situations. However, improved performance did not generalize to the natural school setting and treatment did not effect ratings of peer acceptance. The implications of these findings for future social skills training with children are discussed.     

Correlates of adult and child perceptions of social competency

In this study, the relationship between two commonly used measures of social competency, peer ratings of perceived status and adult ratings of assertiveness, were examined. In Phase I, high and low sociometric status children were videotaped responding to a series of role-play scenes; performance on these scenes was then rated retrospectively by adults for overall assertiveness. In Phase II, a second group of children who were unfamiliar with the videotaped children observed the taped scenes and provided ratings of likability, intelligence, working together, and attractiveness. In general, results revealed that children and adults differed in their ratings of the videotaped children. Further, boys used different cues in their ratings than did girls. More specifically, the sociometric status of girls was related to perceived attractiveness, whereas the sociometric status of boys was more closely related to perceived intelligence. Such findings suggest that children and adults look for and use different cues in their judgments. Given these differences, it is concluded that child and adult ratings should not be used interchangeably in the assessment of, and evaluation of, social competency in children.Portions of this research were supported by a grant from the Virginia Department of Criminal Justice Services.     

Analogue observational methods in the assessment of social functioning in adults

This article provides a clinically oriented overview of analogue observational methods used in the assessment of problematic social functioning, specifically skill deficits and social anxiety. This article emphasizes role-play assessment methods, the predominant method used in clinical settings. An examination of the psychometric characteristics of analogue assessment methods is presented, followed by a review of procedural and structural considerations that may impact the quality of assessment data. Of special concern are the potential impacts of instructional variables, structured versus ideographic role-played situations, confederate characteristics and behavior, molar and molecular levels of assessment, self-ratings versus clinician ratings of functioning, and physical attractiveness. Finally, published and empirically evaluated analogue observation tests are critically reviewed with an emphasis on features that may impact their utility in clinical practice.     

Reliability of heterosocial skills measurement with sex offenders

The heterosocial skills of 79 sex offenders were assessed using two self-report inventories, the Social Anxiety and Distress Scale (SAD) and the S-R Inventory of Anxiousness (SRI). In addition, staff members rated the offenders on modified versions of the SAD and the Heterosocial Skills Behavior Checklist for Males (HSB). Twenty-one offenders were also videotaped during a brief interaction with a female confederate. The tapes were rated using the HSB. All ratings and videotapings were conducted twice within a two-week period. The two self-report scales showed good test-retest reliability (r=.88) and correlatedr=.63 with each other. However, the SAD correlated significantly with the Crowne-Marlowe Social Desirability Scale and showed significant change from first to second administrations. The SRI showed less reactivity to social desirability and changed less over time. The staff ratings showed higher test-retest (r=.72–.76) than interrater (r=.36–.43) reliability. They correlatedr=–.56 with each other and did not show a significant relationship to the Crowne-Marlowe. However, both staff-ratings changed significantly from test to retest. Inspection of correlations between staff and self-report measures indicated that these instruments may be strongly affected by rater variance. Different measures completed by the same person correlated higher than the same measure completed by different persons. The HSB showed high levels of interrater agreement (90–100%) and split-half reliability (r=.95–.97). However, test-retest reliability was poor for the overall score and for the category measuring form of conversation. Categories measuring voice and affect were moderately reliable over time (r=.53–.74). However, the voice category scores changed substantially from first to second videotaping. On almost all measures used in this study, the sex offenders' scores were similar to those derived from normal populations. Only the category measuring affect on the HSB seemed to differentiate this group from normals.     

Stress Inoculation Training For Impulsive Behaviors In Adults With Traumatic Brain Injury

A stress inoculation training program involving relaxation, self-instructional, and coping skills training was delivered to 5 persons with traumatic brain injury in an attempt to reduce the frequency of their impulsive behaviors. A multiple-baseline across subjects design was utilized to evaluate the effectiveness of the 10-week intervention. Dependent measures included frequency of impulsive behaviors as recorded by staff members at the residential facility where all 5 participants resided, ratings of role-play-probes, effectiveness ratings by the participants' rehabilitation counselors, and interview information obtained from the participants and their counselors. The data showed a small, but consistent reduction in impulsive behaviors across the 5 participants. Additionally, results indicated changes in role-play ratings did not correspond to changes in in-vivo behavioral frequency recordings for 4 of the 5 subjects. These findings suggest that individuals with moderate to severe impairments may require more salient behavioral contingencies and consistent cuing procedures to control their impulsive behavior.     

Cognitive control and consistency in compliance

Abstract

The purpose of this prospective study was two-fold. First, three modes of compliance assessment were used to examine whether renal dialysis patients comply consistently across medical regimens (fluid, potassium, phosphorous, protein) and whether compliance is consistent across mode of assessment (patient self assessment, medical staff ratings, physiological data). Second. a cognitive model predicting fluid compliance was tested to see if it would generalize to predict dietary compliance and medication taking. Patients' self-control perceptions of compliance, staff assessments of compliance, and physiological data were collected prospectively for 85 end-stage renal disease (ESRD) patients. Results indicated substantial consistency across medical regimen depending on the mode of assessment; staff assessment showed the most consistency, followed by patients' self-assessments and lastly by physiological data. Despite this consistency across medical regimens, the cognitive-control model only predicted fluid compliance; the model failed to explain dietary and medication compliance. Reasons and implications for these results are discussed.     

Internal consistency and reliability of alternate short forms of the college women's assertion sample: Clinical and research implications

Assertive behavior is most often assessed with self-report or role-play measures. The latter modality is preferred because it provides for the sampling of the structure of behavior and for the consideration of the situational context. MacDonald (1978) has developed such an assessment device but it is limited by the length of time for administration and scoring. Two studies were conducted to reconstruct reliable alternate short forms. The first study describes the selection of items and demonstrates the internal consistency of the alternate forms. The second study demonstrates the alternate form and retest reliability and provides normative statistics. We conclude that reliable alternate short forms have been constructed to be used in research in clinical applications.This research was supported by the Marie Wilson Howells Fund.Alternate short forms of the CWAS may be obtained from the first author.     

The relationship between counsellor interpersonal skills and the core conditions of client-centred counselling

The present study aimed to determine whether there was a systematic relationship between Rogers'(1957) core conditions of empathy, acceptance and genuineness, and counsellors' use of specific behaviours or skills which have been isolated for training. Video recordings of 12 counsellors' performance in 'naturalistic' counselling interactions were analysed. Counsellor, client and judges' ratings of the core conditions were obtained. Skill assessment included molar ratings of eight skill areas and molecular measures of 29 behaviours. The results indicated that there were few significant relationships. It is concluded that there is a discrepancy between specification of the skills or behaviours which are thought to communicate the core conditions and those which have been empirically demonstrated to do so. Implications for counsellor training and proposals for future research are discussed.     

INTERACTIVE VIDEO ASSESSMENT OF CONFLICT RESOLUTION SKILLS

In this research we developed and validated an interactive video assessment of conflict resolution skills. A model of conflict management was used to develop the conflict scenarios and part of the scoring key. Computer assessments of conflict resolution skills and two cognitive abilities were administered to 347 supervisors and job performance ratings were collected from their managers. The conflict skills assessment was found to be significantly related to supervisory ratings of on-the-job performance in managing conflict but to be unrelated to the measures of cognitive ability. In addition, the conflict skills assessment had no adverse impact for women. The implications of these results and directions for future research are discussed.     

Factors Underlying the Psychological and Behavioral Characteristics of Office of Strategic Services Candidates: The Assessment of Men Data Revisited

During World War II, the Office of Strategic Services (OSS), the forerunner of the Central Intelligence Agency, sought the assistance of clinical psychologists and psychiatrists to establish an assessment program for evaluating candidates for the OSS. The assessment team developed a novel and rigorous program to evaluate OSS candidates. It is described in Assessment of Men: Selection of Personnel for the Office of Strategic Services (OSS Assessment Staff, 1948). This study examines the sole remaining multivariate data matrix that includes all final ratings for a group of candidates (n = 133) assessed near the end of the assessment program. It applies the modern statistical methods of both exploratory and confirmatory factor analysis to this rich and highly unique data set. An exploratory factor analysis solution suggested 3 factors underlie the OSS assessment staff ratings. Confirmatory factor analysis results of multiple plausible substantive models reveal that a 3-factor model provides the best fit to these data. The 3 factors are emotional/interpersonal factors (social relations, emotional stability, security), intelligence processing (effective IQ, propaganda skills, observing and reporting), and agency/surgency (motivation, energy and initiative, leadership, physical ability). These factors are discussed in terms of their potential utility for personnel selection within the intelligence community.     

Infant Distress During Immunization: A Multimethod Assessment

The purpose of this study was to comprehensively describe infant procedural distress and pain across assessment modalities, and to compare similarities and differences across measures. A multimethod assessment of distress was conducted to investigate infants (N = 37) undergoing routine immunizations. Measures of infant distress included Parent report, nurse report, infant heart rate, and an observational measure of infant distress. Parents rated their infant's distress and pain significantly higher than did nurses. Observational and physiological ratings of infant distress were found to vary significantly by phase, and there were no correlations between adult ratings of pain and distress and physiological ratings. Findings suggest that infant procedural distress can be assessed in a number of manners. The discordance between these measures emphasizes the need for multimethod assessment of pediatric procedural distress in both research and clinical settings. Given the differences between parent and nurse ratings, clinicians should be aware that different assessment methods might lead to different conclusions about infant procedural distress.     

Personalized versus standard role plays in the assessment of depressed patients' social skill

Chronically depressed outpatients participated in a study of the role-play assessment of social skill. Patients role-played a series of standard situations typical of those employed in social-skills research and additional scenes drawn from critical situations in their personal lives. These “personalized” role-play situations were perceived as more relevant by patients and resulted in more discomfort and less skillful interpersonal behavior than standard scenes. Behavioral measures derived from personalized role plays were also found to be more strongly related to measures of depression than the same measures derived from the standard role-play scenes. Implications of these findings for the role-play assessment of social skill are discussed.     

How to bring the news … peak-end effects in children’s affective responses to peer assessments of their social behavior

The retrospective evaluation of an event tends to be based on how the experience felt during the most intense moment and the last moment. Two experiments tested whether this so-called peak-end effect influences how primary school students are affected by peer assessments. In both experiments, children (ages 7–12) assessed two classmates on their behaviour in school and then received two manipulated assessments. In Experiment 1 (N ?=?30), one assessment consisted of four negative ratings and the other of four negative ratings with an extra moderately negative rating added to the end. In Experiment 2 (N ?=?44), one assessment consisted of four positive ratings, and the other added an extra moderately positive rating to the end. Consistent with the peak-end effect, the extended assessment in Experiment 1 and the short assessment in Experiment 2 were remembered as more pleasant and less difficult to deal with, which shaped children’s peer assessment preferences and prospective choices of which assessment to repeat. These findings indicate that the process of peer assessment can be improved by ending the feedback with the most positive part of the assessment.     

Relating Motivation to Read and Emergent Reading Skills: A Measurement Validation Study

Two studies examined the construct validity of emergent readers' motivation measures. Both studies directly assessed reading skills. Motivation was measured via self-report inventories, including existing and new items derived from the Preschool Reading Attitudes Survey (PRAS), the Motivated Reading Questionnaire (MRQ), and the Motivated Reading Profile (MRP). Parent and teacher measures of emergent readers' motivation were also administered. Study 1 findings indicated significant correlations within readers' self ratings and between teachers' and parents' ratings of children's motivation, but little consistency between these ratings and children's self-ratings. Study 2 replicated many Study 1 findings and indicated stability in children's motivation measures.     

Convergent and concurrent validation of four measures of assertion

This study examined the relationship among the College Self-Expression Scale (CSES), the Dominance Scale of the California Psychological Inventory (Do), role-play assessment, and peer ratings of assertion. Peer ratings served as the criterion measure. Fifty-three members of a college sorority completed the CSES and the Do Scale, role played eight scenes requiring an assertive response, and rated each other on a five-point scale of assertion. Peer ratings were significantly correlated with the Do Scale and the CSES. A significant correlation was also obtained between the CSES and the Do. Role-play ratings were not significantly related to any measure. Results are discussed in terms of convergent and concurrent validity for the Do and CSES as measures of assertion. A precautionary note is presented regarding the use of role-play assessment for measuring assertion and as a behavioral validation procedure.