Using dynamic risk and protective factors to predict inpatient aggression: Reliability and validity of START assessments

The Short-Term Assessment of Risk and Treatability (START; C. D. Webster, M. L. Martin, J. Brink, T. L. Nicholls, & S. L. Desmarais, 2009; C. D. Webster, M. L. Martin, J. Brink, T. L. Nicholls, & C. Middleton, 2004) is a relatively new structured professional judgment guide for the assessment and management of short-term risks associated with mental, substance use, and personality disorders. The scheme may be distinguished from other violence risk assessment instruments because of its inclusion of 20 dynamic factors that are rated in terms of both vulnerability and strength. This study examined the reliability and validity of START assessments in predicting inpatient aggression. Research assistants completed START assessments for 120 male forensic psychiatric patients through review of hospital files. They also completed Historical-Clinical-Risk Management-20 (HCR-20; C. D. Webster, K. S. Douglas, D. Eaves, & S. D. Hart, 1997) and Hare Psychopathy Checklist: Screening Version (PCL:SV; S. D. Hart, D. N. Cox, & R. D. Hare, 1995) assessments. Outcome data were coded from hospital files for a 12-month follow-up period using the Overt Aggression Scale (OAS; S. C. Yudofsky, J. M. Silver, W. Jackson, J. Endicott, & D. W. Williams, 1986). START assessments evidenced excellent interrater reliability and demonstrated both predictive and incremental validity over the HCR-20 Historical subscale scores and PCL:SV total scores. Overall, results support the reliability and validity of START assessments and use of the structured professional judgment approach more broadly, as well as the value of using dynamic risk and protective factors to assess violence risk. (PsycINFO Database Record (c) 2012 APA, all rights reserved).     

Taking “the boss” into the real world: Field interrater reliability of the Short‐Term Assessment of Risk and Treatability: Adolescent Version

There is emerging evidence that the performance of risk assessment instruments is weaker when used for clinical decision‐making than for research purposes. For instance, research has found lower agreement between evaluators when the risk assessments are conducted during routine practice. We examined the field interrater reliability of the Short‐Term Assessment of Risk and Treatability: Adolescent Version (START:AV). Clinicians in a Dutch secure youth care facility completed START:AV assessments as part of the treatment routine. Consistent with previous literature, interrater reliability of the items and total scores was lower than previously reported in non‐field studies. Nevertheless, moderate to good interrater reliability was found for final risk judgments on most adverse outcomes. Field studies provide insights into the actual performance of structured risk assessment in real‐world settings, exposing factors that affect reliability. This information is relevant for those who wish to implement structured risk assessment with a level of reliability that is defensible considering the high stakes.     

Assessing the effectiveness of jail diversion programs for persons with serious mental illness and co-occurring substance use disorders

In recent years, jail diversion programs for people with serious mental illness and co-occurring substance use disorders have received increasing attention and have rapidly grown in number. Previous studies suggest that jail diversion programs have the potential to achieve positive outcomes. The present study reports findings from 6 jail diversion programs (3 pre-booking and 3 post-booking) participating in a federally-funded research initiative to assess the effectiveness of jail diversion programs for people with co-occurring disorders. Diverted and non-diverted groups were compared on self-reported outcomes at 12 months following diversion. The findings suggest that jail diversion reduces time spent in jail without increasing the public safety risk, while linking participants to community-based services. Jail diversion costs and the implications of these results for jail diversion programs and future research are discussed.     

The Short-Term Assessment of Risk and Treatability (START): a prospective validation study in a forensic psychiatric sample

A new assessment scheme--the Short-Term Assessment of Risk and Treatability (START)--presents a workable method for assessing risks to self and others encountered in mentally and personality disordered clients. This study aimed to demonstrate (a) prevalence and severity of risk behaviors measured by the START, (b) psychometric properties of START, (c) similarities and differences in START scores across different mental health professionals, and (d) concurrent validity of START with diverse negative outcomes. Treatment team members completed the 20-item, dynamically focused START for 137 forensic psychiatric inpatients. Prevalence and severity of START risk domains were measured for 51 patients detained in the hospital for 1 year. Results revealed high rates of generally low-level adverse events. With some exceptions, START scores were meaningfully associated with outcomes measured by a modified Overt Aggression Scale.     

A comparative study of adolescent risk assessment instruments: predictive and incremental validity

Promising new adolescent risk assessment tools are being incorporated into clinical practice but currently possess limited evidence of predictive validity regarding their individual and/or combined use in risk assessments. The current study compares three structured adolescent risk instruments, Youth Level of Service/Case Management Inventory (YLS/CMI), Structured Assessment of Violence Risk in Youth (SAVRY), and Psychopathy Checklist: Youth Version (PCL:YV), for both predictive and incremental validity with respect to general and violent recidivism. Receiver operating characteristic and hierarchical logistic regression analyses revealed that the risk tools predicted general and violent recidivism to varying degrees of accuracy, but the SAVRY offered the most in incremental validity. Clinical implications and future directions for youth risk assessment are discussed.     

A multitrait–multimethod study of assessment instruments for complex problem solving

Recently published studies on Complex Problem Solving (CPS) suggest that assessments of CPS using multiple complex systems are only moderately related to tests of classical cognitive abilities. Further, CPS assessments show incremental validity beyond tests of other cognitive abilities when predicting relevant outcomes. However, these empirical accounts have relied on single CPS assessment instruments. We do not know whether these findings will generalize to the construct level across different CPS assessment instruments. To answer this question, we tested a sample of N = 339 German university students who completed three CPS assessment instruments based on multiple complex systems (MicroDYN, the Genetics Lab, and MicroFIN) and the matrices subtest of the Intelligence Structure Test as measure of reasoning. Students further reported their school grades. Analyses including latent multitrait–multimethod models provided support for the conceptualization of CPS as a complex cognitive ability. Results indicated that different CPS assessment instruments showed sufficient convergent validity (with a consistency mostly between .50 and .60). In addition, we found evidence for the divergent validity of CPS from reasoning (reasoning predicted two CPS facets, knowledge and control, β_KNOW = .49 and β_CON = .53, respectively). In the prediction of academic achievement, CPS explained variance in natural science grades after we controlled for reasoning (β_CPS = .22), whereas social science grades were not predicted. Our findings suggest that the validity of CPS generalizes across different measurement instruments.     

Real and ideal self-images in second- and fifth-grade children: Reliability,validity and factorial structure

The reliability, construct validity, and factorial structure of three self-image instruments were assesses in a total of 80 second- and fifth-grade children. Both real self-images (children's current views of themselves) and ideal self-images (the self views to which children aspire) were assessed. the split-half and test—retest reliabilities of the instruments were adequate even for children's as young as second graders. Stronger evidence of construct validity was found for the older children, although the intercorrelation patterns for both age groups were characterized by larger correlations between assessments of real and ideal self-images using the same instrument than between assessments of the same component of the self-image across the three instruments. Contrary to predictions gener ated by developmental theory, the factor analyses provided non evidence of increasing differentiation of the self-image with age. They did, however, reveal that second and fifth graders distinguish distinctly different functional domains within their self-images.     

Critical elements of the crisis intervention team model of jail diversion: An expert survey

The Crisis Intervention Team (CIT) model of jail diversion is a promising approach to addressing the over-involvement of people with mental illness with the criminal justice system. Despite its popularity and promising empirical support, the literature has yet to clarify CIT's critical elements. The aim of this study was to assess the degree to which experts agreement on the importance and perceived implementation of the critical elements of CIT. Study 1 used a literature review to cull potential elements. Three experts familiar with the CIT model reviewed these elements. Study 2 utilized an online survey of 55 CIT co-ordinators/officers and published experts to assess agreement regarding importance and degree the element is implemented. A preliminary list of 36 elements was identified with a high level of expert agreement on their importance regarding importance and implementation. This study developed a preliminary list of elements of the CIT's program. Further research should develop consensus, explore elements' association with outcomes, and differentiate elements from those common to all jail diversion programs. Copyright © 2010 John Wiley & Sons, Ltd.     

Presidential Leadership Style and the Political Use of Force

A growing body of work seeks to explain the lack of clear evidence for the diversionary use of force by casting doubt on such strategies' attractiveness for policy makers: while domestic political and economic problems may provide incentives for diversion, such strategies involve political and military risks that frequently outweigh these incentives. Such theories correctly identify the objective risks involved in diversion but do not account for variation in leaders' risk‐taking propensities. We develop a “first image” theory of diversion that suggests a key psychological variable (locus of control) shapes leaders' willingness to engage in risky diversionary strategies. A statistical analysis of the American use of force, 1953–2000, finds strong support for this model. We conclude that the lack of clear evidence for diversion in general is a reflection of the contingent nature of the phenomenon and call for greater attention to how agents and structures interact to produce policy behavior.     

A short and very short form of the physical self-inventory for adolescents: Development and factor validity

ObjectivesThe Physical Self-Inventory (PSI)—a French adaptation of the Fox and Corbin's [1989. The Physical Self-Perception Profile: Development and preliminary validation. Journal of Sport and Exercise Psychology, 11, 408–430] Physical Self-Perception Profile—was originally developed for use with adults and no study has systematically verified its psychometric properties in adolescent populations. Additionally, this instrument remains too long to be efficiently completed in combination with multiple other instruments within extensive longitudinal or idiographic studies. The purpose of the present investigation was thus threefold: (a) testing the factor validity and reliability of the original PSI in a sample of adolescents; (b) developing and testing the factor validity and reliability of a very short (i.e., two items per scale) form of the PSI in a sample of adolescents; and (c) testing the equivalence of the factor pattern, structural parameters, latent mean structure, and criterion-related validity of both forms of the PSI.DesignStructural equation modeling approach.MethodTwo samples participated in this series of studies. In Study 1, a sample of 1018 adolescents completed the adult PSI (25 items) and was randomly split in two sub-samples. In Study 2, a new sample of 320 adolescents completed a very short form of the PSI (PSI-VSF). Factorial validity and gender and multigroup invariance of these instruments (PSI, PSI-VSF) were tested using confirmatory factorial analysis (CFA) and structural equation modeling (SEM).ResultsIn Study 1, CFA and SEM analyses provided evidence for the factor validity and reliability of a short (PSI-SF: 18 items) and very short (PSI-VSF: 12 items) form of the PSI for adolescents. In Study 2, CFAs and SEMs supported the equivalence of the factor pattern, structural parameters, latent mean structure, and criterion-related validity of both forms of the PSI (i.e., PSI-SF, PSI-VSF).ConclusionThe present results thus provide preliminary evidence regarding the reliability and validity of a short and a very short form of the PSI for French adolescents.     

Psychopathic features in a juvenile diversion population: reliability and predictive validity of two self-report measures

The reliability and predictive validity of two experimental self-report versions of two measures of psychopathic features in youth, the Antisocial Process Screening Device (APSD: Frick & Hare, 2001) and a modified version of the Child Psychopathy Scale (CPS: Lynam, 1997) were examined in a sample of 69 arrested youth (M age = 14.4 years) referred to a juvenile diversion program. Parents or legal guardians completed parent-rating versions of the same measures. Reliability indicators for the APSD total measure were satisfactory although internal consistency indices (coefficient alpha) for the Callous/Unemotional and Impulsive/Conduct Problems scales were slightly lower than desirable. Reliability indicators for the CPS were excellent after deleting items that had poor corrected item-to-CPS total score correlations. Positive and statistically significant correlations for all measures were obtained with prospective measures of program failure (range 0.22-0.36) and rearrest at 1 year follow-up (range 0.33-0.56). Although further research is needed prior to the clinical use of these measures, these results signal the potential of such measures to inform clinical judgments regarding treatment compliance and risk with justice-involved youth.     

Computerized Neurocognitive Testing in the Management of Sport-Related Concussion: An Update

Since the late nineties, computerized neurocognitive testing has become a central component of sport-related concussion (SRC) management at all levels of sport. In 2005, a review of the available evidence on the psychometric properties of four computerized neuropsychological test batteries concluded that the tests did not possess the necessary criteria to warrant clinical application. Since the publication of that review, several more computerized neurocognitive tests have entered the market place. The purpose of this review is to summarize the body of published studies on psychometric properties and clinical utility of computerized neurocognitive tests available for use in the assessment of SRC. A review of the literature from 2005 to 2013 was conducted to gather evidence of test-retest reliability and clinical validity of these instruments. Reviewed articles included both prospective and retrospective studies of primarily sport-based adult and pediatric samples. Summaries are provided regarding the available evidence of reliability and validity for the most commonly used computerized neurocognitive tests in sports settings.     

Cross-cultural validity of integrity assessments for lower-level and higher-level jobs

This research brief examines the cross-cultural validity of two multi-dimensional integrity assessments, one for lower-level jobs and one for higher-level jobs. Job applicants for a global company completed the assessments as a part of the hiring process in three countries: the United States, South Africa, and Mexico. U.S. applicants tended to score higher (i.e., higher integrity) on these assessments and make fewer admissions of counterproductive work behaviors than applicants from South Africa and Mexico. The overall scores for these assessments (and most of the individual scales) were correlated significantly with, for each country separately and globally, three composites of self-reported counterproductive work behavior criterion measures (Theft, Counterproductivity, and Overall Risky Behavior). Construct validity evidence was examined with a measure of Collusion Avoidance (tolerance for colluding with external organized criminals). Overall, the results of this research highlight the importance of including an integrity assessment in the hiring processes for lower-level and higher-level (i.e., professional and managerial) jobs globally. Integrity assessments appear to have practical value in countries with higher prevalence of crime and corruption. The results also bolster previous research indicating that integrity assessments are valid outside of the U.S. context in which they were initially developed over 60 years ago.     

Assessment of methods for measuring social support

A rapidly expanding literature documents the effects of social support on physical and psychological well-being. Although definitions vary, most include both tangible components (e.g., financial assistance and physical aid) and intangible components (e.g., encouragement and guidance). Social support has been implicated in the mediation of stressful life events, recovery from illness, and increased program adherence. There are many inconsistent findings in the literature, however, and it is difficult to resolve discrepancies because measures of social support vary widely from study to study. To guide in the selection of measurement methods for research and applied work, 23 techniques for assessing social support are reviewed and evaluated. Criteria for favorable evaluation included reliability coefficients greater than .8 and documentation of validity. Correlations between various social support and criterion measures are simulated in order to demonstrate the consequences of choosing a measure with low reliability. Scale developers reported reliability data for 19 of the reviewed measures. Internal consistency coefficients ranged from .31 to .98. Test-retest coefficients ranged from .22 to .96. At least some validity documentation was available for 13 of the scales. Discriminant validity evidence, however, is almost universally absent. Despite psychometric weaknesses and variability among the scales, researchers have several instruments available to them.     

Using the Implicit Association Test to Assess Risk Propensity Self‐concept: Analysis of its Predictive Validity on a Risk‐taking Behaviour in a Natural Setting

The present work analyses the predictive validity of measures provided by several available self‐report and indirect measurement instruments to assess risk propensity (RP) and proposes a measurement instrument using the Implicit Association Test: the IAT of Risk Propensity Self‐Concept (IAT‐RPSC), an adaptation of the prior IAT‐RP of Dislich et al. Study 1 analysed the relationship between IAT‐RPSC scores and several RP self‐report measures. Participants' risk‐taking behaviour in a natural setting was also assessed, analyzing the predictive validity of the IAT‐RPSC scores on risk‐taking behaviour compared with the self‐report measures. Study 2 analysed the predictive validity of the IAT‐RPSC scores in comparison with other indirect measures. Results of these studies showed that the IAT‐RPSC scores exhibited good reliability and were positively correlated to several self‐report and indirect measures, providing evidence for convergent validity. Most importantly, the IAT‐RPSC scores predicted risk‐taking behaviour in a natural setting with real consequences above and beyond all other self‐report and indirect measures analysed. Copyright © 2013 European Association of Personality Psychology     

Die Vorhersage der Rückfälligkeit entlassener Sexualstraftäter mittels des Sexual Offender Risk Appraisal Guide (SORAG) und dessen Screening-Version (SORAG-SV): Darstellung der differentiellen und prädiktiven Validität

The Sex Offender Risk Appraisal Guide (SORAG) is one of the most important actuarial risk assessment instruments for sexual offenders. In the meantime, the application of actuarial instruments in recidivism risk assessment for sexual offenders is regarded as state of the art. In addition to results about interrater reliability and concurrent validity we present results about the differential and predictive validity of the SORAG and a screening version of the SORAG (SORAG-SV), which relies only on file information. In order to examine the predictive validity, we used a representative sample of 519 male sexual offenders released from a prison sentence served in one of Austrian´s prisons with a mean follow-up period of 3½ years. Furthermore, we tested the differential validity by dividing the whole sample in different subsamples regarding age, index offense type, and degree of antisociality. Both SORAG and SORAG-SV showed predominantly good predictive accuracy which, however, varied depending on offender subgroup and recidivism category.     

Cognitive ability in virtual reality: Validity evidence for VR game‐based assessments

Virtual reality (VR) technology may provide unique benefits for assessing individual characteristics, in contrast with traditional assessments. This study examined validity evidence for the use of VR game (VRG) scores to indicate specific cognitive abilities. Participants completed three VRGs, three computer‐based assessments (CBAs), and additional scales. To evaluate convergent validity, VRG scores were compared with CBA scores. For divergent validity, VRG scores were compared with five factors of personality based on the Five Factor Model. For criterion‐related validity, VRG scores were compared with academic achievement (i.e., grade point average). Findings generally provided support that the VRG scores are related to certain spatial reasoning subdimensions of cognitive ability. However, further research is needed to examine the qualities of VRGs as assessment tools.     

Editor's Note: Series Coda

There is ongoing discussion in the literature regarding the validity of various checklist and scenario-based measures of guilt-and shame-proneness. Confirmatory factor analysis was used to perform multitrait-multimethod assessments of trait, method, and error components in the top three contending instruments is this area. College students (85 men, 197 women) completed the Test of Self-Conscious Affect-Modified (TOSCA-M) Personal Feelings Questionnaire-2 (PFQ-2), and the Guilt Inventory. A model in which shame and guilt each represented a separate latent trait construct provided a good fit to the data. Scores for shame from both the TOSCA-M and PFQ-2 loaded highly on the latent shame factor. With the exception of scores for nonruminative guilt derived from the TOSCA-M, all remaining scores for guilt had substantial loadings on the latent trait construct of guilt. Discussion focuses on the need to pay serious attention to methods effects in this area, provides explanations for why two qualitatively distinct forms of guilt can be exacted using these methods, and urges specialists in this area to refrain from using single measurement instruments to assess multiple construct.     

Operationalizing the Assessment and Management of Violence Risk in the Short-term

Assessing risk of violence in the short term is crucial for managing and preventing violence, especially in institutions such as psychiatric units and prisons. Despite a lack of consensus on the definition of "short term", a number of recent tools and guidelines have been developed to aid short-term clinical decision-making. Whereas the supporting evidence for the new tools is impressive, limitations remain in terms of the focus on prediction, limited consideration of strengths, and poor integration with formulation and risk management. The Short-Term Assessment of Risk and Treatability (START) is a brief clinical guide for the dynamic assessment of risks, strengths and treatability. It focuses on short-term risks and the characteristics of the individual that, if changed, might lead to an increase or decrease in risk. The START has the potential to operationalize the structured professional judgment (SPJ) approach in order to inform the evaluation of multiple risk domains relevant to everyday psychiatric clinical practice. However, explicit guidance on integrating risk assessment, formulation and management is limited in the START and this paper describes the SPJ approach, reviews recent developments in approaches to risk, and considers how the START can be used to inform SPJ approaches and link risk assessment, formulation, and management. Copyright ? 2012 John Wiley & Sons, Ltd.     

Internal consistency and construct validity of two versions of the Revised Vividness of Visual Imagery Questionnaire

The Vividness of Visual Imagery Questionnaire (VVIQ; Marks, 1973) is the questionnaire most extensively used to measure imagery vividness, but two newer versions have been developed. In this study, the reliability and construct validity of these two measures were assessed: the Vividness of Visual Imagery Questionnaire-2 (VVIQ-2) and the Vividness of Visual Imagery Questionnaire-Revised Version (VVIQ-RV). A total of 206 undergraduate psychology students completed both questionnaires, and to assess construct validity, they also completed the Object-Spatial Imagery and Verbal Questionnaire (OSIVQ), Betts' Questionnaire Upon Mental Imagery (Betts' QMI), and the Measure of Ability to Form Spatial Mental Imagery (MASMI). Internal consistency reliability for both was high, with the VVIQ-RV having a Cronbach's alpha of .96, and the VVIQ-2 an alpha of .91. The construct validity of the VVIQ-RV and the VVIQ-2 was supported by the high correlations they had with the other three measures of imagery, which were all close to .50, and the very small correlations (.06-.07) they had with the Verbal scale of the OSIVQ, indicating that they validly measured the imagery construct. Thus, the findings support the two versions as equally valid psychometric instruments for measuring image vividness.