Assessing treatability in mentally disordered offenders: A conceptual and methodological note

The assessment of treatability in mentally disordered offenders has rarely been addressed through empirical research, despite its frequent use in clinical and legal decision-making. We describe four necessary steps for establishing the requisite research base for treatability evaluations: (a) criterion specification, (b) the development of reliable measures, (c) the validation of these measures with treatment outcome data, and (d) the incorporation of environmental and base rate information.     

A brief measure of attitudes towards qualitative research in psychology

The use and teaching of qualitative research methods in psychology is increasing, but to date no measure has been developed to identify and measure changes in attitudes towards qualitative research in psychology student, academic, and scientist‐practitioner populations. In this article, we present the development and initial validation of a new measure, Attitudes Toward Qualitative Research in Psychology. A pool of 46 items developed from previous qualitative research along with validation measures was administered via an online survey to a convenience sample of 288 psychology students, academics, and psychologists. Principal axis factoring with varimax rotation produced a four‐factor, 18‐item solution. All factors have acceptable internal reliability. Known‐groups validity analyses based on preferred research orientation, and convergent and divergent validity analyses based on measures of attitudes towards quantitative research and researcher/practitioner orientation, provide initial validation of the measure. This brief, internally reliable measure can be used in assessing attitudes towards qualitative research in psychology and measuring change over time.     

Diagnosing organizational cultures: A conceptual and empirical review of culture effectiveness surveys

This review traces the development of survey research methods within the organizational culture tradition and focuses specifically on those instruments that measure the aspects of culture that are related to organizational effectiveness. Our review suggests that the reliability and validity of most instruments in this category is quite limited. This review outlines the recommended logic for the development and validation of culture effectiveness surveys and identifies three key challenges for future culture researchers to address: (1) the confirmatory testing of nested models, (2) the guidelines for aggregating data to the organizational level, and (3) the establishing of criterion-related validity. Using data from the Denison Organizational Culture Survey, we present an empirical illustration of the three challenges identified above and conclude by considering limitations and opportunities for future research.     

SELECTING A COST-EFFECTIVE SCREENING MEASURE FOR THE ASSESSMENT OF PRESCHOOL SOCIAL WITHDRAWAL

The utility of teacher judgment (rankings, ratings) and peer judgment (sociometric ratings) were studied as screening variables for preschool social withdrawal/responsiveness. Observational measures of preschoolers' interaction were used as the validation criterion based upon theoretical, empirical, and practical considerations related to development of a multipurpose behavioral assessment system. Results indicated that teacher rankings of students' verbal interaction frequencies (1) were most highly correlated to interaction rate in two preschool settings, (2) had test-retest reliability consistently above Rho = .80, and (3) were useful in that 77% of teachers using this procedure could identify their least socially responsive student within five rankings. The peer nomination sociometric rating was the least reliable measure at retest (Rho = .35) and showed consistently the lowest correlation with observational indices of interaction partner preferences, e.g., reciprocal rate (Rho = .29 to .39). The tendency for relationships among screening measures and the criterion to increase at retest suggested a training effect on screening accuracy. The use of teacher rankings in combination with observational measures to confirm selection accuracy and for monitoring treatment progress is discussed as a cost-effective behavioral assessment procedure for preschool social withdrawal/responsiveness.     

Development and Initial Validation of the Child Routines Inventory

The importance of establishing predictable routines during childhood consistently has been emphasized in the popular parenting literature, despite a paucity of empirical evidence. The lack of research may be partially due to a lack of suitable instruments designed to measure children's routines. This study describes development and preliminary psychometric data in support of the Child Routines Inventory (CRI), an empirically derived, content-valid, parent-report measure of commonly occurring routines in school-aged children. Principal components analysis yielded a four-factor structure (composed of 36 items), accounting for 44.2% of the total variance. The CRI was found to have excellent internal consistency ( = .90), good test-retest reliability (r = .86), and preliminary evidence of construct validity, demonstrating a moderate relationship with measures of child behavior problems and family routines. These results suggest the CRI to be a promising new measure of child routines that would benefit from further validation studies.     

Development and preliminary validation of the physiological hyperarousal scale for children

Considerable empirical support exists for the positive affect and negative affect components of the tripartite model of anxiety and depression proposed by L. A. Clark and D. Watson (1991); however, less attention has been paid to the physiological hyperarousal component of the model. The development of the Physiological Hyperarousal Scale for Children (PH-C; J. Laurent, S. J. Catanzaro, & T. E. Joiner Jr., 1995) is described. The psychometric properties of items are examined using students in Grades 6-12 (N = 398). Initial scale validation includes a joint factor analysis with the Positive and Negative Affect Scale for Children (PANAS-C; J. Laurent et al., 1999; J. Laurent, K. Potter, & S. J. Catanzaro, 1994). The relationship between the PH-C and existing measures that tap related constructs is examined. Together, the PH-C and PANAS-C provide a means to assess tripartite model constructs useful in differentiating anxiety and depression.     

Boredom proneness--the development and correlates of a new scale

This article reports the development, validation, and correlates of a self-report measure of boredom proneness. The 28-item Boredom Proneness (BP) Scale demonstrates satisfactory levels of internal consistency (coefficient alpha = .79) and test-retest reliability (r = .83) over a 1-week interval. Evidence of validity for the BP is supported by correlations with other boredom measures and from a set of studies evaluating interest and attention in the classroom. Other hypothesized relationships with boredom were tested, with significant positive associations found with depression, hopelessness, perceived effort, loneliness, and amotivational orientation. Additional findings indicate boredom proneness to be negatively related to life satisfaction and autonomy orientation. The relationship of boredom to other affective states is discussed, and directions for future research are outlined.     

The psychodynamic psychiatry service of the University of Alberta Hospital: a thirty year history

This paper describes the Psychodynamic Psychiatry Service (PPS) of the University of Alberta Hospital over its 30 years of development. This psychiatric organization consists of three clinical programs-an outpatient clinic and intensive day and evening programs-and an integral evaluation and research unit. The PPS is unique in its group therapy clinical orientation, its psychodynamic theoretical orientation, and its integration of an ongoing research program that establishes empirical validation of its clinical work. The productivity and longevity of this psychiatric organization appear to derive from several strengths, including cooperation between leaders of the clinical and research programs; the institution of staff relations groups in the three clinical programs; the operation of the fully integrated evaluation and research program that serves to provide empirical support for the treatments offered; and a unifying ideology characterized by the valuing of both psychodynamic and group oriented work. Other important factors to the success of the PPS include the strengths of the founder of the service and financial and other support of the academic department in which it is housed. This paper describes the historical development and present structure and functioning of the PPS, the challenges it has been confronted with, and the responses to those challenges. We conclude with factors contributing to its survival and productivity and with thoughts about the future.     

DEVELOPMENT OF A JOB ANALYSIS-BASED PROCEDURE FOR WEIGHTING AND COMBINING CONTENT RELATED TESTS INTO A SINGLE TEST BATTERY SCORE

By definition, content-related approaches to test validation do not rely on criterion data. As a consequence, regression and other statistical procedures for weighting and generating a composite score from a test battery are not applicable when a content-related validation strategy is used. This paper presents a procedure for determining the component weights for a test battery that has been developed on the basis of a content-related validity strategy. The Relative Content Contribution (RCC) weighting procedure is a logical extension of the conceptual basis underlying the rational developmental process used to demonstrate the validity of content-related tests. Results from field implementations of the procedure in the development of two promotional test batteries (fire and police) and an entry-level test battery (police) in two large metropolitan cities are presented to illustrate the procedure.     

Androgyny and its relation to adolescent psychosocial well-being: A review of the literature

The theoretical and empirical literature on androgyny in adolescence is reviewed in this paper. The traditional bipolar view of sex roles and more recent models of sex role development are contrasted. Then the research literature on the relation between sex role orientation and psychosocial well-being in adolescence is reviewed. From a review of several research articles on the topic, it is concluded that it is masculinity and the masculine component of androgyny that is most associated with both male and female adolescents' psychosocial well-being. This conclusion is discussed in light of current issues in the area of sex roles.     

EXPLORING OPTIONS FOR SUPPORTING TEST USE IN SITUATIONS PRECLUDING LOCAL VALIDATION

This paper examines some of the options available to practitioners interested in supporting the use of selection measures in an organization, including test transportability, validity generalization (VG), and synthetic validation, reviewing some of the advantages, disadvantages and requirements of each approach. Results of four unpublished, proprietary validation studies are reported which compare validity estimates provided by the job component validation (JCV; a type of synthetic validation) routine inherent in the Position Analysis Questionnaire (PAQ) with observed validity coefficients for a variety of criterion measures. We then examine the accuracy of the JCV procedure in predicting validity coefficients for 51 clerical classifications extracted from an existing PAQ job evaluation database of a large utility company. Predicted JCVs are compared to mean observed validity coefficients for five DOT clerical categories provided by Pearlman, Schmidt, and Hunter (1980). The VG and JCV methods provided highly similar and converging estimates of the validity of cognitive ability tests for predicting performance in clerical occupations. Implications for practice are discussed, particularly the need to use multiple, converging lines of evidence to support test use.     

Development and Validation of the Sexist Attitudes Toward Women Stale (SATWS)

Although sexism is now defined as a major social problem, there have been very few empirical studies investigating its correlates and determinants. One primary reason for this is that there are no published sexism scales, and hence it is difficult to assess individual differences in sexism. This article describes the development and validation of a 40-item Sexist Attitudes Toward Women Scale (SATWS). On rational-intuitive grounds, it is hypothesized that sexism toward women has seven components and, accordingly, items were written to reflect these. It is suggested that scales measuring only one of these components (e.g., sex-role stereotype scales and women's liberation movement scales) lack content validity as measures of sexism. On empirical grounds it is argued that the SATW scale also has stronger construct validity as a measure of sexism toward women than other such scales. The SATW scale has high internal consistency reliability for both college students and nonstudent adults and the SATW scale scores are independent of social desirability scores. Relationships to age, sex, and education are presented.     

Optimizing the validity of personality assessments: The importance of aggregation and item content

The predictive validity of a psychological measure can be improved by minimizing measurement errors through increases in the length of the assessment (aggregation) and, for an assessment of finite length, by making use of objective strategies for choosing from all available component measures. Two prominent considerations in selecting individual measures to be aggregated involve standards of (a) item content (construct approach) and (b) item/criterion association (empirical approach). Personality trait scales of different lengths were assembled for this study in order to represent features of the construct and empirical methods of selection. It was observed that (a) although reliability and validity generally increased with test length, aggregation beyond a certain point can fail to be expedient; and (b) although the prediction performance of empirically derived measures initially surpassed that of construct based assessments, the superiority of the empirical scales did not generalize to trait criteria that were not used as a basis for item selection. The data are interpreted as providing support for a theory-based program of test development where substantive considerations involving item content play a major role. The findings are also viewed as encouragement for conventional conceptualizations about organized dimensions of behavior.     

Toward an assessment of the construct validity of four measures of narcissism

In recent years there has been growing conceptual interest in narcissism, coupled with the rapid development of several paper and pencil measures. Clearly, construct validation should be performed prior to the adoption of any measure. This study reports on the construct validation of four scales that purport to measure narcissism. A conceptual definition of narcissism is provided, and a number of empirical analyses are performed, including examination of internal consistency reliabilities, intercorrelations among the four narcissism scales, and correlations with related and unrelated measures. Three of the four scales were found to be consistently related to each other, and demonstrated similar patterns of substantive and discriminant validity. A factor analysis, performed to examine the structure of the four narcissism measures, suggests four dimensions: desire for attention, admiration, and authority; feelings of inadequacy, unhappiness, and worry; disturbances in interpersonal relationships (exploitation, sense of entitlement, lack of empathy); and sensitivity to criticism and defeat. Implications of the results are discussed.     

Development and validation of the Physical Appearance Related Teasing Scale

The development and preliminary validation of the Physical Appearance Related Testing Scale (PARTS) is reported. The 18-item scale has good psychometric qualities (high internal consistency and test-retest reliabilities) and consists of two factors: Weight/Size Testing (W/ST) and General Appearance Teasing (GAT). The W/ST converges well with measures of eating disturbance, body image dissatisfaction, social comparison, depression, and self-esteem, whereas the GAT scale shows little relationship to these variables. The role of teasing history as an etiological component of body image and eating disturbance is discussed; the need for further validation work on the PARTS is addressed.     

Development and Validation of the Physical Appearance Related Teasing Scale

The development and preliminary validation of the Physical Appearance Related Testing Scale (PARTS) is reported. The 18-item scale has good psychometric qualities (high internal consistency and test-retest retiabilities) and consists of two factors: Weight/Size Testing (W/ST) and General Appearance Teasing (GAT). The W/ST converges well with measures of eating disturbance, body image dissatisfaction, social comparison, depression, and self-esteem, whereas the GAT scale shows little relationship to these variables. The role of teasing history as an etiological component of body image and eating disturbance is discussed; the need for further validation work on the PARTS is addressed.     

Active versus passive learning and testing in a complex outside built environment

A review of the evidence on active and passive learning in virtual environments (VEs) suggests that both conditions have shown superiority under some conditions of learning and testing, but there is no consistent outcome pattern. Measures of transfer between virtual and real environments have also revealed a variety of outcomes. Following either active or passive learning in a VE, experiment 1 assessed measures of orientation and distance estimation in that VE and in a real-world equivalent environment. On measures of direct and relative distance, more accurate estimates were found for active than passive VE explorers. A suggestion was also noted for the orientation estimates to benefit from real-world rather than VE testing. With an improvement to the procedure, experiment 2 found similar real versus virtual orientation judgements, suggesting that an opportunity for active learning during the test procedure probably influenced orientation measures in experiment 1. We conclude that the effects of interactivity are unreliable and vary with the measures used, and that testing in virtual and real environments leads to similar outcomes.     

Option weights should be determined empirically and not by experts when assessing knowledge with multiple‐choice items

Multiple‐choice tests are frequently used in personnel selection contexts to measure knowledge and abilities. Option weighting is an alternative multiple‐choice scoring procedure that awards partial credit for incomplete knowledge reflected in applicants’ distractor choices. We investigated whether option weights should be based on expert judgment or on empirical data when trying to outperform conventional number‐right scoring in terms of reliability and validity. To obtain generalizable results, we used repeated random sub‐sampling validation and found that empirical option weighting, but not expert option weighting, increased the reliability of a knowledge test. Neither option weighting procedure improved test validity. We recommend to improve the reliability of existing ability and knowledge tests used for personnel selection by computing and publishing empirical option weights.     

Measuring children's regulation of emotion-expressive behavior

Emotion regulation has become a pivotal concept in developmental and clinical research. However, the measurement of regulatory processes has proved extremely difficult, particularly in the context of within-subject designs. Here, we describe a formal conceptualization and a new experimental procedure, the Balloons Game, to measure a regulatory component of emotion-expressive behavior. We present the internal consistency and stability of the indices derived from the Balloons Game in a sample of 121 kindergarten children. External validation against measures that have been associated with emotion regulation processes is also provided. The findings suggest that the Balloons Game provides a reliable tool for the study of regulation of emotion expression in young children.     

Introduction to the special section on incremental validity and utility in clinical assessment

This special section focuses on the incremental validity and utility of clinical assessment data. The lack of replicated incremental validity research limits the ability of psychologists to establish their assessment practices on a solid empirical footing. The articles in this section deal with conceptual, methodological, and content issues in the development and use of clinical measures. The authors addressed several aspects of incremental validity research, including (a) the evaluation of the magnitude of validity increments, (b) the use of incremental validity data for test development and validation, (c) the costs of assessment, (d) the use of multi-informant and multimethod assessments, and (e) the treatment utility of assessment. Obstacles limiting the use of incremental validity research to inform and guide clinical practice are also emphasized.