Mixed group validation: a method to address the limitations of criterion group validation in research on malingering detection

Mixed group validation (MGV) is offered as an alternative to criterion group validation (CGV) to estimate the true positive and false positive rates of tests and other diagnostic signs. CGV requires perfect confidence about each research participant's status with respect to the presence or absence of pathology. MGV determines diagnostic efficiencies based on group data; knowing an individual's status with respect to pathology is not required. MGV can use relatively weak indicators to validate better diagnostic signs, whereas CGV requires perfect diagnostic signs to avoid error in computing true positive and false positive rates. The process of MGV is explained, and a computer simulation demonstrates the soundness of the procedure. MGV of the Rey 15-Item Memory Test (Rey, 1958) for 723 pre-trial criminal defendants resulted in higher estimates of true positive rates and lower estimates of false positive rates as compared with prior research conducted with CGV. The author demonstrates how MGV addresses all the criticisms Rogers (1997b) outlined for differential prevalence designs in malingering detection research.     

Detecting symptom exaggeration in combat veterans using the MMPI-2 symptom validity scales: a mixed group validation

Although validity scales of the Minnesota Multiphasic Personality Inventory-2 (MMPI-2; J. N. Butcher, W. G. Dahlstrom, J. R. Graham, A. Tellegen, & B. Kaemmer, 1989) have proven useful in the detection of symptom exaggeration in criterion-group validation (CGV) studies, usually comparing instructed feigners with known patient groups, the application of these scales has been problematic when assessing combat veterans undergoing posttraumatic stress disorder (PTSD) examinations. Mixed group validation (MGV) was employed to determine the efficacy of MMPI-2 exaggeration scales in compensation-seeking (CS) and noncompensation-seeking (NCS) veterans. Unlike CGV, MGV allows for a mix of exaggerating and nonexaggerating individuals in each group, does not require that the exaggeration versus nonexaggerating status of any individual be known, and can be adjusted for different base-rate estimates. MMPI-2 responses of 377 male veterans were examined according to CS versus NCS status. MGV was calculated using 4 sets of base-rate estimates drawn from the literature. The validity scales generally performed well (adequate sensitivity, specificity, and efficiency) under most base-rate estimations, and most produced cutoff scores that showed adequate detection of symptom exaggeration, regardless of base-rate assumptions. These results support the use of MMPI-2 validity scales for PTSD evaluations in veteran populations, even under varying base rates of symptom exaggeration.     

Further validation of the IDAS: evidence of convergent, discriminant, criterion, and incremental validity

The authors explicated the validity of the Inventory of Depression and Anxiety Symptoms (IDAS; D. Watson et al., 2007) in 2 samples (306 college students and 605 psychiatric patients). The IDAS scales showed strong convergent validity in relation to parallel interview-based scores on the Clinician Rating version of the IDAS; the mean convergent correlations were .51 and .62 in the student and patient samples, respectively. With the exception of the Well-Being Scale, the scales also consistently demonstrated significant discriminant validity. Furthermore, the scales displayed substantial criterion validity in relation to Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association, 1994) mood and anxiety disorder diagnoses in the patient sample. The authors identified particularly clear and strong associations between (a) major depression and the IDAS General Depression, Dysphoria and Well-Being scales, (b) panic disorder and IDAS Panic, (c) posttraumatic stress disorder and IDAS Traumatic Intrusions, and (d) social phobia and IDAS Social Anxiety. Finally, in logistic regression analyses, the IDAS scales showed significant incremental validity in predicting several DSM-IV diagnoses when compared against the Beck Depression Inventory-II (A. T. Beck, R. A. Steer, & G. K. Brown, 1996) and the Beck Anxiety Inventory (A. T. Beck & R. A. Steer, 1990).     

A method for testing group differences of scale validity in multiple population studies

A method for testing equality in validity of multi‐component measuring instruments across populations is outlined. The approach is developed within the framework of covariance structure modelling and complements earlier research on examining group differences in scale reliability. The procedure is particularly useful for purposes of ascertaining comparability of validity when constructing and developing measuring instruments. The method also provides ranges of plausible values for differences in composite validity across several populations and allows one to evaluate group discrepancies in validity of behavioural scales. The approach is illustrated using data from a cognitive intervention study.     

A criterion for determining the optimum value of twist in the topological model

In previous works, in which the topological model has been applied to martensitic phase transformations, the value of twist angle ω was determined based on the habit plane-(HP) matching method, where the physical realization of the so-predicted interfacial defect networks may require reorientations of defect line directions by short-range diffusion, though no long-range diffusion was needed. In the present work, a novel criterion for determining the optimum value of twist is proposed so that the predicted interface defects are not only able to fulfil the function of fully accommodating the coherency strains arising on the terrace plane, but also capable of reaching the required position at the HP without long- or short-range diffusions. A numerical analysis for an Fe–20Ni–5Mn alloy is demonstrated based on the newly proposed criterion, and the predictions so obtained are in good agreement with the results provided by the phenomenological theory and experimental measurements.     

A test of the validity of the IS PATH WARM warning signs for suicide

The 10 warning signs for suicide, encapsulated in the mnemonic IS PATH WARM, did not differentiate between genuine suicide notes and those written by people simulating a suicidal crisis or between notes written by completed suicides and those written by attempted suicides.     

Research validity scales for the NEO-PI-R: development and initial validation

The purpose of this investigation was to develop a set of research validity scales for use with the NEO Personality Inventory-Revised (NEO-PI-R; Costa & McCrae, 1992b). In study 1 we used the existing NEO-PI-R item pool to select items for three validity scales: positive presentation management, negative presentation management, and inconsistency. Several iterative item and scale analyses were conducted, using multiple criteria for item selection. These analyses resulted in 10-item scales. In Study 2, the internal consistency, interscale relationships, and normative characteristics were examined in a separate sample of working adults. In Study 3, the validity of the scales was examined by contrasting five sets of NEO-PI-R protocols: from a separate sample of working adults, from a sample of 100 NEO-PI-R protocols with randomly produced responses, and from three samples of undergraduates completing the NEO-PI-R under different instructional sets. Analyses revealed that both the research validity scales and the NEO-PI-R domain scales were sensitive to group differences. issues relating to the appropriate use of the scales are discussed.     

A method for determining the aversive threshold in the rat using repeated measures: Tests with morphine sulfate

The procedure described here provides an objective automated technique for the psychophysical assessment of the aversive threshold and allows the animal Ss to serve as their own controls. Sensitivity to drug effects, as well as comparisons between drugs, using the same animals are possible also. Using a rectangular tilt cage, the aversive threshold to grid shock was defined as that intensity avoided 75% of the time. Each shock intensity (30, 60, 90, 120, and 150 microA) was presented for 5 min on one side of the cage and then switched to the other side for 5 min, thereby forcing the animal to sample each shock intensity. Aversive thresholds were determined daily for each rat. The analgesic effects of morphine sulfate were compared to saline. Animals were tested for 3 days at each morphine dose level (2, 4, 8, and 16 mg/kg). Each 3-day morphine series alternated with 3 days of testing under saline. Significant differences were detected between saline and morphine at 4, 8, and 16 mg/kg.     

EMG waveform duration: A validation method for the surface electromyogram

The electromyogram (EMG) recorded with surface electrodes was examined using a triggered oscilloscope technique. When all but the high-amplitude waveforms in a given segment of activity were excluded from analysis, it was found that the duration of the remaining waveforms was specific to the muscle that produced them. That is, waveforms of a given duration were recorded whenever conventional indices-observable contraction, joint movement. needle recordings-showed activity in a specific muscle. This led to the suggestion that waveform duration might be used to validate the surface EMG. The suggestion. which is limited by the need to control posture and electrode placement, both of which may influence waveform duration, was illustrated in application to psychological research.     

Evaluating the predictive validity of personality trait judgments using a naturalistic behavioral criterion: A preliminary test of the self-other knowledge asymmetry model

We tracked 87 participants over two days using the Electronically Activated Recorder (EAR). Coded variables included expressions of mood, amount of talking in various situations (e.g., with one other person, with a friend, etc.), locations, and behavioral markers of the Big Five. Collection of self-, informant-, and stranger-ratings on markers of the Big Five allowed for a unique test of the Self-Other Knowledge Asymmetry (SOKA) model. Although effect sizes were modest, there was evidence for the validity of both self- and informant-ratings across most trait dimensions. Stranger-ratings showed evidence of validity in the domain of Extraversion. Predictions derived from the SOKA model were partially supported, though more research with larger samples is needed to provide stronger tests of SOKA.     

Independence and nonindependence: A simple method for comparing groups using multiple measures and the binomial test

Existing methods for conducting analyses of small group data are either highly complicated or yield low power. Both of these limitations provide disincentives for the progress of research in this field. An alternative method modelled on the sign (binomial) test which involves comparing the differences of distributions based on multiple observations of each of the groups is presented. The calculations involved in the procedure are extremely simple. It is suggested that because the method enhances researchers' ability to make sound statistical inferences easily this should stimulate research on group‐level processes and on social interaction more generally. Copyright © 2004 John Wiley & Sons, Ltd.     

Comparison of decision learning models using the generalization criterion method

It is a hallmark of a good model to make accurate a priori predictions to new conditions ( Busemeyer & Wang, 2000 ). This study compared 8 decision learning models with respect to their generalizability. Participants performed 2 tasks (the Iowa Gambling Task and the Soochow Gambling Task), and each model made a priori predictions by estimating the parameters for each participant from 1 task and using those same parameters to predict on the other task. Three methods were used to evaluate the models at the individual level of analysis. The first method used a post hoc fit criterion, the second method used a generalization criterion for short-term predictions, and the third method again used a generalization criterion for long-term predictions. The results suggest that the models with the prospect utility function can make generalizable predictions to new conditions, and different learning models are needed for making short-versus long-term predictions on simple gambling tasks.     

Item synonymization: A method for determining the total meaning of pencil-paper reactions

Items have been studied heretofore for their value as elements of particular tests to the neglect of more fundamental research into the multiple potentiality of items. This article proposes a method of grouping items into synonymies comprising all of the items which correlate with a given key item. These synonymies can be used for interpretation of the total meaning of the key item: (1) by inspection of the constituent items and (2) by correlational study of obtained single scores of individual persons. The method is illustrated by four items with inter- and intra-correlations, and characteristics of an ideal background reservoir of items are pointed out.     

A revised method for projecting two different movies to two groups of viewers without their noticing the duality

The MORI technique reported in Mori (2003) has evolved into its second generation after the appearance of a new type of video projector with two advantages over the first generation. The new MORI technique uses a pair of DLP projectors instead of the LCD type. The improved technique does not require sideways framing of the movie for presentation as required formerly. Therefore, any ready-made movie clip can be used. Second, it utilizes a pair of circular polarizing filters instead of linear polarizers. Linear polarizing filters may leak the to-be-blocked light wave when they are tilted. Circular polarizing filters are less susceptible to that weakness. The effectiveness of the revised technique has been proved by several studies carried out in two separate laboratories in which undergraduates and preschoolers participated as eyewitnesses to observe simulated criminal events presented using this technique.     

Evidence for the criterion validity and clinical utility of the Pathological Narcissism Inventory

In this study, the authors evaluated aspects of criterion validity and clinical utility of the grandiosity and vulnerability components of the Pathological Narcissism Inventory (PNI) using two undergraduate samples (N = 299 and 500). Criterion validity was assessed by evaluating the correlations of narcissistic grandiosity and narcissistic vulnerability with established indices of normal personality traits, psychopathology and clinical concerns, and pathological personality traits. Overall, the pattern of correlations supported the convergent and discriminant validity of grandiose and vulnerable conceptualizations of pathological narcissism as measured by the PNI. Clinical utility was assessed by evaluating the extent to which clinicians without specific training in pathological narcissism as well as clinicians with expertise in pathological narcissism could accurately predict the correlates of PNI grandiosity and vulnerability with normal and pathological personality traits and psychopathology. The r(contrast-cv) coefficient provided a global index of accuracy in clinicians' predictions that was more fully elaborated by examining systematic discrepancies across groups. Overall, novice and expert clinicians were generally able to predict criterion correlations, with some exceptions (e.g., counter to predictions, pathological narcissism was negatively associated with treatment resistance). These results provide further evidence regarding the validity and utility of the narcissistic grandiosity and narcissistic vulnerability constructs as measured by the PNI.     

Assessment of performance validity in the Stroop Color and Word Test in mild traumatic brain injury patients: A criterion‐groups validation design

The current study assessed performance validity on the Stroop Color and Word Test (Stroop) in mild traumatic brain injury (TBI) using criterion‐groups validation. The sample consisted of 77 patients with a reported history of mild TBI. Data from 42 moderate–severe TBI and 75 non‐head‐injured patients with other clinical diagnoses were also examined. TBI patients were categorized on the basis of Slick, Sherman, and Iverson (1999) criteria for malingered neurocognitive dysfunction (MND). Classification accuracy is reported for three indicators (Word, Color, and Color–Word residual raw scores) from the Stroop across a range of injury severities. With false‐positive rates set at approximately 5%, sensitivity was as high as 29%. The clinical implications of these findings are discussed.     

An examination of the construct validity of posttraumatic stress disorder with veterans using a revised criterion set

Ongoing concerns exist in the literature regarding the construct of posttraumatic stress disorder (PTSD) and how to best conceptualize and measure this disorder. We compared the traditional DSM-IV PTSD symptom criteria (i.e., symptoms from clusters B, C, and D) to a revised criterion set that omits overlapping mood and other anxiety symptoms on PTSD prevalence, PTSD diagnostic caseness, associated psychiatric comorbidity, functional status, and structural validity using a cross-sectional, multi-site primary care sample of 747 veterans. After removing items theorized to overlap with mood and other anxiety disorders, PTSD prevalence was identical using both criterion sets (i.e., 12%). Overall, there were few statistically significant differences in PTSD caseness, associated psychiatric comorbidity, functional status, and structural validity across the two diagnostic criterion sets. These data provide further support that removing items that overlap with other psychiatric disorders does not significantly impact the prevalence of PTSD, its associated comorbidity and functional impairment, or its structural validity. Although the revised criterion set represents a more parsimonious model, the current study findings generally support the strong construct validity of PTSD. The implications of these study findings for research and clinical practice are discussed.