Factors affecting the social validity of a role-play test of children's social skills

The social validity of a role-play test of children's social skills, the Social Skills Test for Children (SST-C), was evaluated by 63 subjects (33 adults and 30 children) who subjectively judged the quality of three children's role-play performances. The three children were selected as having high, medium, or low social skill based upon their performance on the SST-C. Assessment of the effects of subject characteristics upon judgments of social skill suggested that similarities in age and race between the subjects and the three children tended to diminish perceived differences in social skill among the three children. It was concluded that the social validity of the role-play test was supported only if the persons evaluating social behavior were dissimilar in age and racial characteristics from the persons who were being evaluated.     

Differential validity by sex in employment settings

The current study meta-analyzes 59 male-female pairs of validity coefficients obtained during revalidation of the General Aptitude Test Battery (GATB) for evidence of differential validity by sex. Results suggest that general cognitive ability may be slightly more predictive of rated job performance for women than for men in our sample of studies. Moderator analyses examined the effects of occupational sex-dominance and cognitive complexity. Results of these analyses indicate that, in single sex-dominated occupations, tests of general cognitive ability and perceptual ability are more valid for members of the dominant sex, and these differences are more apparent at low-complexity levels. Explanations for the findings are offered in terms of properties of the ratings criterion.An earlier version of this paper was given at the 2nd annual convention of the Society for Industrial and Organizational Psychology, Inc., Atlanta, GA. The authors thank Dave Synk for providing the occupation codes for each study.     

Trends in the measurement of social validity

Since its inception in the mid-1970s, social validity has provided applied behavior analysts with a critical measure of the social impact and importance of their interventions. Recent discussion, however, has questioned the use of this construct in regard to the frequency and types of social validty measures employed in research. Despite the ensuing discussion, virtually no quantitative information has been made available to frame various perspectives and opinions. The purpose of this report is to present a content analysis of social validity measures used over the previous 20 years. Social validity was assessed along three dimensions: (a) type of assessment, (b) focus of assessment, and (c) time of assessment. Articles published in the Journal of Applied Behavior Analysis (1968-1990) and Behavior Modification (1977-1990) were surveyed. The results of the content analysis indicate that current applications of social validation procedures are presented in 20% of the articles surveyed. The majority of articles used subjective evaluation of outcomes following intervention to assess social validity. In addition, the data indicated that normative comparison was a rarely used method of social validation and that its use has been decreasing over time.     

Evidence for the external validity of a social simulation measure of social skills

Two studies were conducted in order to obtain evidence regarding the external validity of a social simulation measure of social skills. The first study employed a contrasted group design, with the groups consisting of National Guardsmen and psychiatric patients. As predicted, trained judges rated the performances of the National Guardsmen in the simulations as more skillful and appearing less anxious than the psychiatric patients. In the second study, trained judges' ratings of psychiatric patients' performances in the simulations were found to be significantly related to ratings made by interviewers who had conducted a structured social history interview with the patients, nurses' ratings based on their observations of patients after 3 days on the psychiatric unit, a research assistant's ratings based on contact with the patient during the experiment, and finally, the subject's own self-ratings. In discussing the findings, it is hypothesized that one reason why support was found for the external validity of the simulation strategy in this study, in contrast to some previously reported studies, was the use of molar ratings in this study as opposed to the more molecular rating procedures used in those previous studies.These studies were funded in part by a research grant from the Veterans Administration.     

Estimating the validity of the guilty knowledge test from simulated experiments: the external validity of mock crime studies

This experiment was designed to examine the external validity of the standard mock-crime procedure used extensively to evaluate the validity of polygraph tests. The authors manipulated the type of mock-crime procedure (standard vs. a more realistic version) and the time of test (immediate vs. delayed) and examined their effects on the validity of the Guilty Knowledge Test (GKT) and the recall rate of the relevant items. The results indicated that only the type of mock-crime affected the 2 outcome variables. The realistic procedure was associated with a lower recall rate and weaker detection efficiency than the standard procedure. However, these effects were mediated by the type of GKT questions used. Practical implications of these results are discussed.     

The impact of corrections for faking on the validity of noncognitive measures in selection settings

In selection research and practice, there have been many attempts to correct scores on noncognitive measures for applicants who may have faked their responses somehow. A related approach with more impact would be identifying and removing faking applicants from consideration for employment entirely, replacing them with high-scoring alternatives. The current study demonstrates that under typical conditions found in selection, even this latter approach has minimal impact on mean performance levels. Results indicate about .1 SD change in mean performance across a range of typical correlations between a faking measure and the criterion. Where trait scores were corrected only for suspected faking, and applicants not removed or replaced, the minimal impact the authors found on mean performance was reduced even further. By comparison, the impact of selection ratio and test validity is much larger across a range of realistic levels of selection ratios and validities. If selection researchers are interested only in maximizing predicted performance or validity, the use of faking measures to correct scores or remove applicants from further employment consideration will produce minimal effects.     

The social validity of treatments for agoraphobia

Two experiments evaluated the public's (Experiment 1) and agoraphobics' ratings of Acceptability and Perceived Effectiveness of five different procedures that have been used to treat agoraphobia. The results showed that both groups evaluated psychological treatment procedures as more acceptable and generally more effective, especially in the long run, than the drug treatments. An exposure treatment was highly rated by both groups, but the two groups differed in their evaluations of a cognitive or a relationship therapy. Recommendations were made for considering the opinions of different consumer groups when evaluating treatment validity.     

The predictive validity of the Structured Assessment of Violence Risk in Youth in secondary educational settings

Current developments in violence risk assessment warrant consideration for use within educational settings. Using a structured professional judgment (SPJ) model, the present study investigated the predictive validity of the Structured Assessment of Violence in Youth (SAVRY) within educational settings. The predictive accuracy of the SAVRY scales was assessed using a retrospective file review to gather data on 87 adolescents ranging in age from 12 to 18 years. Receiver-operating characteristic analyses were used to gauge the predictive accuracy. With an area under the curve of .72 (p = .001), the accuracy of the SAVRY total score in correctly identifying violent youth exceeds the accuracy of identifications based on chance predictions in this sample. Logistic regression analyses assessed the relative contribution of the SAVRY subscales, whereas the omnibus equation using all subscale scores correctly classified 82% of those adolescents who were nonviolent and 45% of those adolescents who were violent. These results build on previous research and provide support for the use of the SAVRY in educational settings for identification as well as directing intervention efforts. Practical implications and areas for future research are also discussed.     

Classroom-based cognitive-behavioral intervention to prevent aggression: Efficacy and social validity

Classroom teachers need effective, efficient strategies to prevent and/or ameliorate destructive student behaviors and increase socially appropriate ones. During the past two decades, researchers have found that cognitive strategies can decrease student disruption/aggression and strengthen pro-social behavior. Following preliminary pilot work, we conducted a study to determine whether a classwide, social problem-solving curriculum affected measures of knowledge and behavior for 165 4th and 5th grade students at risk for behavior problems. We found significant positive treatment effects on knowledge of problem-solving concepts and teacher ratings of aggression. Outcomes differed across teachers/classrooms, and there was no evidence that booster lessons affected treatment efficacy. Teacher ratings of social validity were generally positive. We discuss issues about classroom-based prevention research and future research directions.     

Post-decision consolidation: group members in different social settings compared to individual decision makers

Two experiments were conducted to examine how making decisions in a 3-member group affects the degree of post-decision consolidation, defined as attractiveness changes in favor of the chosen alternative. Both experiments were conducted in two sessions one week apart. In the first session (decision phase), participants estimated the importance of each of four different attributes describing two decision alternatives. They were then shown predetermined attractiveness ratings on each of the attributes and were to decide on the two alternatives, either individually or as a group. In the second session (post-decision phase), the participants were to attempt to recall the attractiveness ratings for the attributes as presented to them in session one. In both experiments, significant post-decision consolidation was found for individual decision makers but not for group members. In experiment 2, this result was replicated, no difference in consolidation being found between group members with face-to-face interaction and without.     

A longitudinal exploration of the continuity of children's social participation and social withdrawal across socioeconomic status levels and social settings

The major purpose of this study was to determine the stability across time and social setting of children's social participation and social withdrawal. We followed a sample of Canadian children, aged five years at the beginning of the study, for two years. Children from lower‐SES homes were less involved than those from the higher‐SES in group play after school and during outdoor recess. Across the full sample, social withdrawal was not very stable. There was, however, considerable stability across setting and time among extreme groups of withdrawn participants. Social withdrawal was largely unrelated to parents' initiation of peer contacts for their children. Copyright © 2000 John Wiley & Sons, Ltd.     

Offender interviewing in custodial settings: Understanding validity,value and veracity issues

The purpose of this scoping literature review was to identify trends regarding validity, value and veracity issues when conducting (qualitative) interviews with offenders in custodial settings. We selected for review works from electronic databases in SAGE, Springer, and Taylor and Francis by using e-journals, Google Scholar and the behavioural science database, PsycINFO. The inclusion criteria for our studies were based on whether sources addressed offender interviewing as a wellspring of information, but excluded those that focused solely on the acquisition of clinical information (at the expense of focusing on validity, value and veracity issues). We utilised the content analysis approach to identify themes/ patterns in the research findings. Key findings suggest that offenders with impression management motivations deceive and manipulate in their research interview presentations. When gathering data from inmates in custodial care settings, best practice includes cross-checking offender interview data against other sources of evidence and taking into account the context in which they are serving their sentences.     

Behavioral intervention to treat selective mutism across multiple social situations and community settings

We evaluated a behavioral intervention for a 9-year-old girl with selective mutism. The intervention consisted of role play and video self-modeling. The frequency of spoken initiations, responses to questions, and communication breakdowns was measured during three social situations (i.e., ordering in a restaurant, meeting new adults, and playing with new children) and in three community settings. Results demonstrated increases in spoken initiations and responses and decreases in communication breakdowns across all situations and settings.     

Autobiographical memories of televised sporting events watched in different social settings

The present study asked 157 men and 154 women to recall their memories of viewing both enjoyed and nonenjoyed televised sporting events from the past year in each of four social circumstances: with friends, with family, with spouse, boyfriend/girlfriend or significant other, and by oneself. For memories of all eight events, participants were asked (1) the number and sex of coviewers, (2) concurrent behaviors and cognitions engaged in while viewing, (3) uses and gratifications sought for enjoyed events, and (4) failed uses and gratifications for nonenjoyed events. Two analyses of variance, one full multiple regression, and one step-wise regression were conducted. Analyses indicated that social circumstance of viewing and sex of participant were associated with uses and gratifications for enjoyed events, failed uses and gratifications for nonenjoyed events, and concurrent behaviors and cognitions engaged in during the viewing. Results are discussed in light of the uses and gratifications paradigm and Wenner and Gantz's 1998 five fanship dimensions. An extension of this model to nonenjoyed sporting events is proposed.     

An assessment of social validity trends in applied behavior analysis

Research articles published during the first 31 years of Journal of Applied Behavior Analysis were assessed for reports of social validity measures. Two measures of social validity (treatment outcome and treatment acceptability) were assessed for each article. In addition, articles were assessed for the incorporation of analog versus naturalistic settings. Overall, treatment outcome and acceptability measures were reported in less than 13% of articles. In addition, studies conducted in naturalistic settings were more likely to report social validity measures than studies conducted in analog settings. Possible reasons for the deficit in the reporting of social validity measures and potential outcomes of this deficit are discussed. Copyright © 1999 John Wiley & Sons, Ltd.     

Motivational underpinnings of social influence in work settings: Bases of social power and the need for cognitive closure

This research explored the notion that the use and efficacy of influence tactics launched from different social power bases depends on influence agents' and recipients' need for cognitive closure. In three separate studies conducted in diverse organizational contexts, it was found that, while overall participants exhibited a preference for soft over hard social influence tactics, this preference becomes less pronounced for supervisors high (versus low) on need for closure and becomes more pronounced for supervisors low on the need for closure. Overall, soft tactics were more beneficial for subordinates' performance than hard tactics; however, the benefits of soft tactics decreased as a function of subordinates' need for closure. Finally, organizational outcomes were improved when recipients who were high (versus low) on the need for closure were exposed to “hard” power tactics and those low (versus high) in the need for closure were exposed to “soft” tactics. Copyright © 2011 John Wiley & Sons, Ltd.