The critical role of the research question, inclusion criteria, and transparency in meta-analyses of integrity test research: a reply to Harris et al. (2012) and Ones, Viswesvaran, and Schmidt (2012)

We clear up a number of misconceptions from the critiques of our meta-analysis (Van Iddekinge, Roth, Raymark, & Odle-Dusseau, 2012). We reiterate that our research question focused on the criterion-related validity of integrity tests for predicting individual work behavior and that our inclusion criteria flowed from this question. We also reviewed the primary studies we could access from Ones, Viswesvaran, and Schmidt's (1993) meta-analysis of integrity tests and found that only about 30% of the studies met our inclusion criteria. Further, analyses of some of the types of studies we had to exclude revealed potentially inflated validity estimates (e.g., corrected validities as high as .80 for polygraph studies). We also discuss our experience trying to obtain primary studies and other information from authors of Harris et al. (2012) and Ones, Viswesvaran, and Schmidt (2012). In addition, we address concerns raised about certain decisions we made and values we used, and we demonstrate how such concerns would have little or no effect on our results or conclusions. Finally, we discuss some other misconceptions about our meta-analysis, as well as some divergent views about the integrity test literature in general. Overall, we stand by our research question, methods, and results, which suggest that the validity of integrity tests for criteria such as job performance and counterproductive work behavior is weaker than the authors of the critiques appear to believe.     

Perceived relevance of honesty and agreeableness to situations with non‐correspondent and correspondent outcomes: An interdependence perspective

According to interdependence theory, interpersonal situations that vary in their surface characteristics can be united by similarities in their underlying structure. Likewise, factor analytic approaches to personality combine many traits into a small number of factors. In the current research, we use interdependence theory and existing factor analyses of personality traits to measure people's lay theories about the ways traits and situations interact. We predict that traits representative of honesty–humility/virtue will be rated as more relevant to situations with non‐correspondent outcomes (a gain in one person's outcomes is associated with a loss in the other person's outcomes) than to situations with correspondent outcomes (a gain in one person's outcomes is associated with a gain in the other person's outcomes). Conversely, we predict that traits representative of agreeableness will be rated as more relevant to situations with correspondent outcomes than to situations with non‐correspondent outcomes. An experiment found the expected trait X situation interaction revealing that subjects expect certain types of traits to be most relevant to specific types of situations. Copyright © 2013 John Wiley & Sons, Ltd.     

“We're not friends anymore! unless…”: the frequency and harmfulness of indirect,relational, and social aggression

The frequency of items of indirect, relational, social, verbal, and physical aggression was assessed in the school environment of 422 adolescents, using the Indirect/Social/Relational Aggression scale (ISRA), a measure that combined items from indirect, relational, and social aggression research. We also assessed the perceived harmfulness of each item. Comparing these findings with the occurrence of aggression on television, we found that adolescents were exposed to nearly 10 times more indirect, relational, and social aggression on television than they are in school. Overall, there was no sex difference in the amount of aggression reported by boys and girls. However, when examining specific items, girls reported more gossiping and boys more hitting. Girls perceived indirect, direct relational, and verbal aggression as more harmful than did boys. Limited evidence was found for a distinction between indirect, relational, and social aggression, although it was clear that they were more similar than different. Aggr. Behav. 32:1–14, 2006. © 2006 Wiley‐Liss, Inc.     

Explicating the Openness to Experience Construct and its Subdimensions and Facets in a Work Setting

Recent meta‐analyses (e.g. Barrick, Mount, & Judge, 2001 ) found openness to experience, a factor in the five‐factor model of personality, to be uncorrelated with job performance. We argue that, among others, insignificant validity is due to the broad and heterogeneous nature of the construct. In line with our hypotheses, we found internal structure of openness to be multidimensional. Further analyses on subdimensional and facet level revealed large differences in criterion‐ and construct‐related validity. It could be demonstrated that a subdimension labeled epistemic curiosity and, especially, the facet openness to ideas, which includes aspects like curiosity, flexibility, willingness to learn, and creativity, are highly relevant for work‐related criteria and so far understudied in organizational research.     

Person‐factors in the California Adult Q‐Set: closing the door on personality trait types?

To investigate recent hypotheses of replicable personality types, we examined data from 1540 self‐sorts on the California Adult Q‐Set (CAQ). Conventional factor analysis of the items showed the expected Five‐Factor Model (FFM). Inverse factor analysis across random subsamples showed that none of the previously reported person‐factors were replicated. Only two factors were replicable, and, most importantly, these factors were contaminated by mean level differences in item endorsement. Results were not due to sample size or age heterogeneity. Subsequent inverse factor analysis of standardized items revealed at least three replicable factors; when five person‐factors were extracted, they could be aligned precisely with the dimensions of the FFM. The major factors of person similarity can be accounted for entirely in terms of the FFM, consistent with the hypothesis that there are no replicable personality types in the CAQ. Published in 2006 John Wiley & Sons, Ltd.     

Malawian secondary students’ beliefs about intelligence

Students who view intelligence as malleable tend to be more academically motivated and perform at higher levels than students who view it as a fixed trait. We examined the beliefs of students from Malawi because the culture and schooling process in this country is very different from some other areas of the world in which students’ views of intelligence have already been studied. Our research questions were: (1) How do Malawian students define intelligence? (2) To what extent do Malawian students view intelligence as malleable? (3) Are Malawian students’ definitions of intelligence and beliefs about the malleability of intelligence similar to those of students in more developed countries? We conducted a mixed methods study and surveyed 136 students attending a secondary school in Malawi using a 39‐item questionnaire. Students responded to questions about their intelligence beliefs on open‐ and closed‐ended items. Our results showed that Malawian students believe that an intelligent student exhibits a variety of behaviors, including studying, working hard, reading, performing well on exams and in class, answering and asking questions, paying attention, and demonstrating good behavior. Most students believe that intelligence is malleable and provided responses that indicated that students can become more intelligent through effort. When compared to the findings of other studies, the present results suggest that the Malawian students who remain in secondary school have definitions of intelligence and beliefs about the malleability of intelligence that are similar to those of students in more developed countries, such as the US and Germany. In fact, it appears that Malawian secondary students have even higher malleable beliefs than American and German students. Finally, some of the measures that have been found to produce scores that are reliable and valid in other populations do not produce scores that are as reliable when used with Malawian students.     

Opinion‐based group membership as a predictor of commitment to political action

Research on group identification has shown it to be a surprisingly weak predictor of intentions to take large‐scale social action. The weak links may exist because researchers have not always examined identification with the type of group that is most relevant for predicting action. Our focus in two studies (one in Romania and one in Australia, both Ns = 101) was on opinion‐based groups (i.e. groups formed around shared opinions). We found that social identification with opinion‐based groups was an excellent predictor of political behavioural intentions, particularly when items measuring identity certainty were included. The results provide clear evidence of the role of social identity constructs for predicting commitment to social action and complement analyses of politicised collective identity and crowd behaviour. Copyright © 2006 John Wiley & Sons, Ltd.     

Evaluating the Factor Structure of the MIDI Personality Scale Using Exploratory Structural Equation Modeling

This study investigated the factor structure of the 26‐item Midlife Development Inventory (MIDI) Personality Scale in a sample of 2,720 Americans. It was found that whereas confirmatory factor analysis (CFA) did not provide an acceptable fit to the data, exploratory structural equation modeling (ESEM) provided an acceptable fit. The results of ESEM revealed that the a priori five‐factor structure of personality was generally consistent with the data, and all items had salient loadings on their target factors. ESEM also revealed that some of the items contributed significantly to more than one personality factor. The results are in line with previous research, and indicate that ESEM is more suitable than CFA for the study of personality traits.     

Coping with bullying at school: children's recommended strategies and attitudes to school‐based interventions in England and Japan

School bullying in England and ijime in Japan have long but separate research traditions. We focus on a cross‐national comparison of secondary school pupils' opinions about coping strategies, bystander intervention, and attitudes towards school‐based interventions. One‐to‐one structured interviews were conducted with 61 Japanese and 60 English pupils aged 12–15 years, in six secondary schools. Coping strategy recommended was found to vary by type of bullying. Seeking help was the most recommended, with significant national differences notably for social exclusion. Victims were thought not to seek help for several reasons, again varying by nationality. There was consensus that bystanders should help victims, but more pupils in England had positive views on school‐based interventions than pupils in Japan. Gender differences were small. Findings are discussed in the light of differing peer group structures and actions taken against bullying/ijime in the two countries. Aggr. Behav. 32:570–580, 2006. © 2006 Wiley‐Liss, Inc.     

Should I believe this? Reality monitoring of accounts of self‐experienced and invented recent and distant autobiographical events

To examine personal and interpersonal reality monitoring, 240 participants wrote accounts of invented or self‐experienced autobiographical events. Half the participants wrote about a distant event that happened before the age of 15 and half wrote about a recent event that happened after the age of 15. Using a yoked design, participants rated the qualitative details of their own accounts and the details of other participants' accounts. Consistent with previous research, we found that self‐experienced accounts contained more qualitative details than invented accounts, and that accounts of recent events contained more qualitative details than accounts of distant events. Participants rated their own accounts as more qualitatively detailed than other participants' accounts, which suggests that they did not base their self‐ratings solely on the written details. We discuss the practical and theoretical importance of our findings. Copyright © 2006 John Wiley & Sons, Ltd.     

Adding Value to Second-Language Listening and Reading Subscores: Using a Score Augmentation Approach

This study examined whether reporting subscores for groups of items within a test section assessing a second-language modality (specifically reading or listening comprehension) added value from a measurement perspective to the information already provided by the section scores. We analyzed the responses of 116,489 test takers to reading and listening items from operational administrations of two large-scale international tests of English as a foreign language. To “strengthen” the reliability of the subscores, and thus improve their added value, we applied a score augmentation method (Haberman, 2008). In doing so, our aim was to examine whether reporting augmented subscores for specific groups of reading and listening items could improve the added value of these subscores and consequently justify providing more fine-grained information about test taker performance. Our analysis indicated that in general, there was lack of support for reporting subscores from a psychometric perspective, and that score augmentation marginally improved the added value of the subscores. We discuss several implications of our findings for test developers wishing to report more fine-grained information about test performance. We conclude by arguing that research on how to best report such refined feedback should remain the focus of future efforts related to second-language proficiency tests.     

Low working memory capacity is only spuriously related to poor reading comprehension

Accounts of comprehension failure, whether in the case of readers with poor skill or when syntactic complexity is high, have overwhelmingly implicated working memory capacity as the key causal factor. However, extant research suggests that this position is not well supported by evidence on the span of active memory during online sentence processing, nor is it well motivated by models that make explicit claims about the memory mechanisms that support language processing. The current study suggests that sensitivity to interference from similar items in memory may provide a better explanation of comprehension failure. Through administration of a comprehensive skill battery, we found that the previously observed association of working memory with comprehension is likely due to the collinearity of working memory with many other reading-related skills, especially IQ. In analyses which removed variance shared with IQ, we found that receptive vocabulary knowledge was the only significant predictor of comprehension performance in our task out of a battery of 24 skill measures. In addition, receptive vocabulary and non-verbal memory for serial order—but not simple verbal memory or working memory—were the only predictors of reading times in the region where interference had its primary affect. We interpret these results in light of a model that emphasizes retrieval interference and the quality of lexical representations as key determinants of successful comprehension.     

The visual metacognition questionnaire: a measure of intuitions about vision

Recent research has revealed a series of striking limits to visual perception. One important aspect of these demonstrations is the degree to which they conflict with intuition; people often believe that they will be able to see things that experiments demonstrate they cannot see. This metacognitive error has been explored with reference to a few specific visual limits, but no study has yet explored people's intuitions about vision more generally. In this article we present the results of a broad survey of these intuitions. Results replicate previous overestimates and underestimates of visual performance and document new misestimates of performance in tasks that assess inattention blindness and visual knowledge. We also completed an initial exploratory factor analysis of the items and found that estimates of visual performance for well-structured information tend to covary. These results represent an important initial step in organizing the intuitions that may prove important in a variety of settings, including performance of complex visual tasks, evaluation of others people's visual experience, and even the teaching of psychology.     

Analyse Préliminaire des Qualités Psychométriques d’une Version Française du Work Design Questionnaire

The Work Design Questionnaire (Morgeson & Humphrey, 2006) assesses numerous work characteristics. The present research was designed to assess the psychometric properties of a French version of this tool (i.e., WDQ-F) through the following analyses: confirmatory factor analysis, reliability, and nomological validity with work satisfaction, engagement and burnout. In Studies 1 and 2, results of confirmatory factor analyses showed that the fit indices for the 21-dimension model were poor and that seven items displayed low factor loadings. In Study 2, results showed that a reduced version of the WDQ-F (71 items) demonstrated better psychometric properties than the 77-item version of the questionnaire. In addition, correlation analyses provided support for the nomological validity of the WDQ-F.     

Emotion's influence on memory for spatial and temporal context

Individuals report remembering emotional items vividly. It is debated whether this report reflects enhanced memory accuracy or a bias to believe emotional memories are vivid. We hypothesised emotion would enhance memory accuracy, improving memory for contextual details. The hallmark of episodic memory is that items are remembered in a spatial and temporal context, so we examined whether an item's valence (positive, negative) or arousal (high, low) would influence its ability to be remembered with those contextual details. Across two experiments, high-arousal items were remembered with spatial and temporal context more often than low-arousal items. Item valence did not influence memory for those details, although positive high-arousal items were recognised or recalled more often than negative items. These data suggest that emotion does not just bias participants to believe they have a vivid memory; rather, the arousal elicited by an event can benefit memory for some types of contextual details.     

Cognitive control in number magnitude processing: evidence from eye-tracking

The unit-decade compatibility effect describes longer response times and higher error rates for incompatible (e.g., 37_52) than compatible (e.g., 42_57) number comparisons. Recent research indicated that the effect depends on the percentage of same-decade filler items. In the present study, we further examined this relationship by recording participants’ eye-fixation behaviour. In four conditions, participants had to compare item sets with different filler item types (i.e., same-decade and same-unit filler items) and different numbers of same-decade filler items (i.e., 25, 50, and 75 %). We found a weaker unit-decade compatibility effect with most fixations on tens in the condition with same-unit filler items. Moreover, the compatibility effect increased with the percentage of same-decade filler items which was accompanied by less fixations on tens and more fixations on units. Thus, our study provides first eye-tracking evidence for the influence of cognitive control in number processing.     

Clinician testimony in suicide litigation: A cause to be uneasy

This is an illustrative article rather than a research study. We offer opinions and recommendations about what we view as unfortunate clinician testimony in suicide‐related malpractice cases, testimony that – inadvertently or not – supports or encourages inadequate care of suicidal patients. The principles apply to both psychiatrists and non‐psychiatrists, although the former appear more often in our work. We particularly consider the roles and testimony, in court or at deposition, of psychiatrists, whether as defendants, expert witnesses, or fact witnesses. We cite examples of what we view as poor, disingenuous, dishonest and even dangerous testimony that we believe moves the profession toward unsafe patient care. The examples illustrate what we (and sometimes others) describe as normalization of deviance, pre‐suit puffery, self‐serving defendant testimony, expert pride supplanting testimonial responsibility, expert arrogance, expert parroting of attorney suggestions, witness ignorance and avoiding facts, unconscious expert bias, inexperience thwarting justice, misleading use of terms such as “predictability,” and expert witnesses who lack the direct‐care experience that jurisdictions often require in order to opine about defendant clinicians' day‐to‐day patient care. The examples often reveal concerns beyond the category chosen, and should not be expected to convey all of the facts of a particular case.     

Emotion's influence on memory for spatial and temporal context

Individuals report remembering emotional items vividly. It is debated whether this report reflects enhanced memory accuracy or a bias to believe emotional memories are vivid. We hypothesized emotion would enhance memory accuracy, improving memory for contextual details. The hallmark of episodic memory is that items are remembered in a spatial and temporal context, so we examined whether an item's valence (positive, negative) or arousal (high, low) would influence its ability to be remembered with those contextual details. Across two experiments, high-arousal items were remembered with spatial and temporal context more often than low-arousal items. Item valence did not influence memory for those details, although positive high-arousal items were recognized or recalled more often than negative items. These data suggest that emotion does not just bias participants to believe they have a vivid memory; rather, the arousal elicited by an event can benefit memory for some types of contextual details.     

Assessing PTSD in the military: Validation of a scale distributed to Danish soldiers after deployment since 1998

Since 1998, soldiers deployed to war zones with the Danish Defense (≈31,000) have been invited to fill out a questionnaire on post‐mission reactions. This provides a unique data source for studying the psychological toll of war. Here, we validate a measure of PTSD‐symptoms from the questionnaire. Soldiers from two cohorts deployed to Afghanistan with the International Security Assistance Force (ISAF) in 2009 (ISAF7, N = 334) and 2013 (ISAF15, N = 278) filled out a standard questionnaire (Psychological Reactions following International Missions, PRIM) concerning a range of post‐deployment reactions including symptoms of PTSD (PRIM‐PTSD). They also filled out a validated measure of PTSD‐symptoms in DSM‐IV, the PTSD‐checklist (PCL). We tested reliability of PRIM‐PTSD by estimating Cronbach's alpha, and tested validity by correlating items, clusters, and overall scale with corresponding items in the PCL. Furthermore, we conducted two confirmatory factor analytic models to test the factor structure of PRIM‐PTSD, and tested measurement invariance of the selected model. Finally, we established a screening and a clinical cutoff score by application of ROC analysis. We found high internal consistency of the PRIM‐PTSD (Cronbach's alpha = 0.88; both cohorts), strong item‐item (0.48–0.83), item‐cluster (0.43–0.72), cluster‐cluster (0.71–0.82) and full‐scale (0.86–0.88) correlations between PRIM‐PTSD and PCL. The factor analyses showed adequate fit of a one‐factor model, which was also found to display strong measurement invariance across cohorts. ROC curve analysis established cutoff scores for screening (sensitivity = 1, specificity = 0.93) and clinical use (sensitivity = 0.71, specificity = 0.98). In conclusion, we find that PRIM‐PTSD is a valid measure for assessing PTSD‐symptoms in Danish soldiers following deployment.     

‘Grandpa,Do You Like Roller Coasters?’: Identifying Age‐Appropriate Personality Indicators

Personality development research heavily relies on the comparison of scale means across age. This approach implicitly assumes that the scales are strictly measurement invariant across age. We questioned this assumption by examining whether appropriate personality indicators change over the lifespan. Moreover, we identified which types of items (e.g. dispositions, behaviours, and interests) are particularly prone to age effects. We reanalyzed the German Revised NEO Personality Inventory normative sample (N = 11,724) and applied a genetic algorithm to select short scales that yield acceptable model fit and reliability across locally weighted samples ranging from 16 to 66 years of age. We then examined how the item selection changes across age points and item types. Emotion‐type items seemed to be interchangeable and generally applicable to people of all ages. Specific interests, attitudes, and social effect items—most prevalent within the domains of Extraversion, Agreeableness, and Openness—seemed to be more prone to measurement variations over age. A large proportion of items were systematically discarded by the item‐selection procedure, indicating that, independent of age, many items are problematic measures of the underlying traits. The implications for personality assessment and personality development research are discussed. © 2019 European Association of Personality Psychology