Nonlinear Personality–Performance Relationships and the Spurious Moderating Effects of Traitedness

Abstract This article evaluated the validity of two traditional traitedness measures (i.e., standard deviation traitedness measures and global, single‐item traitedness measures) and three new traitedness measures (i.e., multi‐item, ranking, and facet traitedness measures). Overall, there was poor construct validity for both the traditional and new measures of traitedness. Furthermore, traitedness was a spurious moderator of the relationship between openness to experience and academic performance. Finally, evidence of nonlinear personality‐academic performance relationships was discovered.     

Optimizing the validity of personality assessments: The importance of aggregation and item content

The predictive validity of a psychological measure can be improved by minimizing measurement errors through increases in the length of the assessment (aggregation) and, for an assessment of finite length, by making use of objective strategies for choosing from all available component measures. Two prominent considerations in selecting individual measures to be aggregated involve standards of (a) item content (construct approach) and (b) item/criterion association (empirical approach). Personality trait scales of different lengths were assembled for this study in order to represent features of the construct and empirical methods of selection. It was observed that (a) although reliability and validity generally increased with test length, aggregation beyond a certain point can fail to be expedient; and (b) although the prediction performance of empirically derived measures initially surpassed that of construct based assessments, the superiority of the empirical scales did not generalize to trait criteria that were not used as a basis for item selection. The data are interpreted as providing support for a theory-based program of test development where substantive considerations involving item content play a major role. The findings are also viewed as encouragement for conventional conceptualizations about organized dimensions of behavior.     

Introducing the Expectations and Preference Scales for Couple Therapy (EPSCT): Development,Psychometric Evaluation,and Suggested Use in Practice and Research

While it is known that client factors account for the largest proportion of outcome variance across treatment modalities, little is known about how clients’ characteristics affect the process and effectiveness of couple therapy. To further knowledge in this area, we created a brief, practice‐friendly measure, the Expectation and Preference Scales for Couple Therapy (EPSCT). Three self‐report scales assess clients’ Outcome expectations (e.g., I expect our relationship to improve as a result of couple therapy) and role expectations for Self (e.g., I expect to listen to my partner's concerns) and Partner (e.g., I expect my partner to blame me). Three Cognitive‐Behavioral, Emotionally Focused, and Family Systems preference scales use a forced‐choice format to measure the comparative strength of respondents’ preferences for interventions broadly reflective of each approach. A large item pool was developed from relevant literature and clinical experience and refined based on face and content analyses with two panels of experienced couple therapists and researchers. Across four studies with 1,175 participants, the scales’ internal consistency reliabilities were similar and their construct validity was supported with confirmatory factor analyses and significant correlations with several established measures, including expectation measures developed for individual psychotherapy and measures of attitudes toward professional help seeking and valuing personal growth. Across all studies, participants had stronger role expectations for themselves than their partners, although gender effects differed by sample. We discuss how to use the 15‐item EPSCT in clinical practice and in future research as a predictor of couple therapy processes and outcomes.     

Examining the validity of self‐reports on scales measuring students' strategic processing

Background. Self‐report inventories trying to measure strategic processing at a global level have been much used in both basic and applied research. However, the validity of global strategy scores is open to question because such inventories assess strategy perceptions outside the context of specific task performance. Aims. The primary aim was to examine the criterion‐related and construct validity of the global strategy data obtained with the Cross‐Curricular Competencies (CCC) scale. Additionally, we wanted to compare the validity of these data with the validity of data obtained with a task‐specific self‐report inventory focusing on the same types of strategies. Sample. The sample included 269 10th‐grade students from 12 different junior high schools. Methods. Global strategy use as assessed with the CCC was compared with task‐specific strategy use reported in three different reading situations. Moreover, relationships between scores on the CCC and scores on measures of text comprehension were examined and compared with relationships between scores on the task‐specific strategy measure and the same comprehension measures. Results. The comparison between the CCC strategy scores and the task‐specific strategy scores suggested only modest criterion‐related validity for the data obtained with the global strategy inventory. The CCC strategy scores were also not related to the text comprehension measures, indicating poor construct validity. In contrast, the task‐specific strategy scores were positively related to the comprehension measures, indicating good construct validity. Conclusion. Attempts to measure strategic processing at a global level seem to have limited validity and utility.     

Assessing the validity of inferences from scores on the cognitive reflection test

Decision‐making researchers purport that a novel cognitive ability construct, cognitive reflection, explains variance in intuitive thinking processes that traditional mental ability constructs do not. However, researchers have questioned the validity of the primary measure because of poor construct conceptualization and lack of validity studies. Prior studies have not adequately aligned the analytical techniques with the theoretical basis of the construct, dual‐processing theory of reasoning. The present study assessed the validity of inferences drawn from the cognitive reflection test (CRT) scores. We analyzed response processes with an item response tree model, a method that aligns with the dual‐processing theory in order to interpret CRT scores. Findings indicate that the intuitive and reflective factors that the test purportedly measures were indistinguishable. Exploratory, post hoc analyses demonstrate that CRT scores are most likely capturing mental abilities. We suggest that future researchers recognize and distinguish between individual differences in cognitive abilities and cognitive processes.     

Using Indirect Measurement Tasks to Assess the Self‐concept of Personality: A Systematic Review and Meta‐analyses

This systematic review presents the current state of research investigating the implicit self‐concept of personality. First, we present results on meta‐analyses estimating internal consistency, reliability coefficients, the implicit–explicit consistency and the single association predictive effect of implicit self‐concept of personality measures. To do this, studies were aggregated over personality domains. Second, for each of the Five Factor personality domains, different aspects of construct validity and predictive validity are reviewed in a narrative way. Results show that implicit self‐concept of personality measures are reliable, and there is evidence for the construct and predictive validity of these implicit measures, especially in the extraversion and agreeableness domains of personality. However, it must be kept in mind that clear evidence for publication bias was found for studies examining the single association predictive pattern. Finally, this systematic review identifies some achievable improvements that are needed in future research. Large cross‐lab efforts are important in this respect. Moreover, the implicit self‐concept of personality field must move from an ‘ad hoc’ to a ‘validation’ approach in developing new indirect measurement tasks. By adopting these research objectives, the information processing account of personality will increase its potential to become integrated into mainstream personality theory and research. Copyright © 2017 European Association of Personality Psychology     

Assessing the validity of multinomial models using extraneous variables: an application to prospective memory

The class of multinomial processing tree (MPT) models has been used extensively in cognitive psychology to model latent cognitive processes. Critical for the usefulness of a MPT model is its psychological validity. Generally, the validity of a MPT model is demonstrated by showing that its parameters are selectively and predictably affected by theoretically meaningful experimental manipulations. Another approach is to test the convergent validity of the model parameters and other extraneous measures intended to measure the same cognitive processes. Here, we advance the concept of construct validity (Cronbach & Meehl, 1955 ) as a criterion for model validity in MPT modelling and show how this approach can be fruitfully utilized using the example of a MPT model of event-based prospective memory. For that purpose, we investigated the convergent validity of the model parameters and established extraneous measures of prospective memory processes over a range of experimental settings, and we found a lack of convergent validity between the two indices. On a conceptual level, these results illustrate the importance of testing convergent validity. Additionally, they have implications for prospective memory research, because they demonstrate that the MPT model of event-based prospective memory is not able to differentiate between different processes contributing to prospective memory performance.     

Measuring creative thinking: An activity‐based approach

The primary objective of this exploratory study was to test activity‐based behavioral measures of creative thinking with a sample of Hong Kong fifth‐grade school children, and also to determine the concurrent validity between activity‐based measures of creative thinking and standard divergent thinking tests. Altogether five creative thinking abilities were measured using the behavioral techniques and the children's scores for fluency, flexibility, and originality were compared with those from the Torrance Test of Creative Thinking (TTCT). The construct‐related validity among all the different abilities on the behavioral measures was also calculated. Contrary to previous assertions about the lack of creativity among Chinese populations, the Hong Kong children demonstrated their ability to use a number of creative thinking features in order to solve problems. No construct validity was found between the different measures of creativity except between originality and fluency. Possible reasons for this are reviewed. The TTCT and the behavioral techniques were found to have some concurrent validity in relationto fluency and originality on the Verbal Tests. There was no correlation for flexibility. The advantages of using activity‐based measurements of creative behavior are discussed.     

Generalization of Cognitive and Noncognitive Validities across Personality‐based Job Families

The positive relationship between complexity of work and the validity of general mental ability (GMA) measures across a variety of occupations is well supported by research and provides important practical and theoretical support for cognitive ability measures. However, there is currently no research demonstrating a systematic relationship between the size of the validities of any personality measure and the personality requirements of jobs, thus leaving open to question the predictive and construct validity of personality measures for applicant selection. We compared the validities of two biodata measures – one scored to measure social competence and one to measure GMA – across six job families that varied in social requirements. The validity of the GMA measure generalized at approximately the same level across the job families while the validity of the social competence measure decreased as social requirements diminished.     

When two sources of fluency meet one cognitive mindset

Fluency, the subjective experience of ease associated with information processing, has been shown to affect a host of judgments. Previous research has typically focused on specific factors that affect the use of a single, specific fluency source. In the present study we examine how cognitive mindsets, or processing modes, moderate fluency emanating from two simultaneous sources of fluency. As a cognitive mindset manipulation, participants performed Jacoby's process dissociation paradigm. Subsequently, participants engaged in a metamemory task that incorporates (and can separately measure the influence of) two simultaneous sources of fluency: familiarity and accessibility. Our results confirmed that our content-unrelated mindset procedure had affected the use of fluency. Moreover, the use of both fluency sources was attenuated, demonstrating the generality of the effect. The findings highlight the causal dependency of fluency on a complex, cognitively-rich environment.     

An evaluation of empirical measures of source identification

Source identification refers to memory for the origin of information. A consistent nomenclature is introduced for empirical measures of source identification which are then mathematically analyzed and evaluated. The ability of the measures to assess source identification independently of identification of an item as old or new depends on assumptions made about how inconsistencies between the item and source components of a source-monitoring task may be resolved. In most circumstances, the empirical measure that is used most often when source identification is measured by collapsing across pairs of sources (sometimes called “the identification-of-origin score”) confounds item identification with source identification. Alternative empirical measures are identified that do not confound item and source identification in specified circumstances. None of the empirical measures examined provides a valid measure of source identification in all circumstances.     

Development of the Spiritual Transcendence Index

In response to calls from the scientific community for improved conceptualizations and measures of spirituality, we describe the concept of "spiritual transcendence" and the development of the Spiritual Transcendence Index (STI). Spiritual transcendence refers to a perceived experience of the sacred that affects one'sself–perception, feelings, goals, and ability to transcend one'sdifficulties. The STI is an eight–item measure of this construct developed and refined based on rational considerations, input from three focus groups of religious leaders, and a reciprocal process of empirically testing the item pool and revising it across six successive surveys. The final scale, together with measures of related constructs, was tested in 220 randomly selected community residents. Although further study of the scale is needed, the STI demonstrated high consistency and validity across several samples in these initial exploratory studies.     

The Tromsø Social Intelligence Scale, a self-report measure of social intelligence

Social intelligence is a construct that not only appeals to laymen as a relevant individual difference but also has shown promising practical applications. Nevertheless, the use of social intelligence in research and applied settings has been limited by definitional problems, difficulties in empirically differentiating social intelligence from related constructs, and the complexity of most existing measures of social intelligence. The goal of the present research was to address some of these obstacles by designing a multi‐faceted social intelligence measure that is short and easy to administer. Three studies were conducted to develop and validate the Tromsø Social Intelligence Scale (TSIS). Study 1 examined professional psychologists’ interpretations of social intelligence to derive a consensually agreed‐upon definition of the construct. In Study 2, a large pool of social intelligence items were tested, and a 3‐factor, 21‐item scale was identified. In Study 3, the stability of this measure was confirmed.     

Factor Analysis of the Mystical Experience Questionnaire: A Study of Experiences Occasioned by the Hallucinogen Psilocybin

A large body of historical evidence describes the use of hallucinogenic compounds, such as psilocybin mushrooms, for religious purposes. But few scientific studies have attempted to measure or characterize hallucinogen‐occasioned spiritual experiences. The present study examined the factor structure of the Mystical Experience Questionnaire (MEQ), a self‐report measure that has been used to assess the effects of hallucinogens in laboratory studies. Participants (N = 1,602) completed the 43‐item MEQ in reference to a mystical or profound experience they had had after ingesting psilocybin. Exploratory factor analysis of the MEQ retained 30 items and revealed a four‐factor structure covering the dimensions of classic mystical experience: unity, noetic quality, sacredness (F1); positive mood (F2); transcendence of time/space (F3); and ineffability (F4). MEQ factor scores showed good internal reliability and correlated with the Hood Mysticism Scale, indicating convergent validity. Participants who endorsed having had a mystical experience on psilocybin, compared to those who did not, had significantly higher factor scores, indicating construct validity. The four‐factor structure was confirmed in a second sample (N = 440) and demonstrated superior fit compared to alternative models. The results provide initial evidence of the validity, reliability, and factor structure of a 30‐item scale for measuring single, hallucinogen‐occasioned mystical experiences, which may be a useful tool in the scientific study of mysticism.     

CALLING: THE DEVELOPMENT OF A SCALE MEASURE

This study clarifies the definition of calling—a consuming, meaningful passion people experience toward a domain—and develops a 12‐item scale measure of calling. Drawing on multiwave longitudinal, 2‐wave longitudinal, and cross‐sectional data from 1,500 participants (2,278 observations) in 4 separate domains, music, art, business, and management, we demonstrate the scale's reliability and unidimensional structure across contexts and over time. We establish the scale's convergent validity and discriminant validity. We determine criterion‐related validity through the scale's relationship to satisfaction with the calling domain, career‐related self‐efficacy, clarity of professional identity, career insight, attending a calling‐oriented college program, professional pursuit of the calling domain, and differences across the 4 domains. We discuss implications of this reliable, valid measure for theory and research on calling, meaning of work, and careers.     

Maternal Sensitivity: Evidence of Stability across Time,Contexts, and Measurement Instruments

Although maternal sensitivity is widely understood as having a direct impact on the quality of the mother–child relationship, empirical evidence supporting this has been inconsistent, possibly because of varied definitions of the construct and limitations of measurements. The Maternal Behavior Q‐Set (MBQS) appears to be a promising measure of sensitivity, showing associations with attachment security (r = .60) higher than those reported (r = .24) in a highly‐regarded meta‐analysis of other sensitivity measures. The present study aimed to further establish the validity and utility of the MBQS by exploring its consistency with a contingency‐based measure of sensitivity, while also seeking evidence of stability across contexts and times. Seventy‐two mothers and their 10‐month‐olds participated in a procedure that included two episodes of Floor Play and Reunions following maternal differential treatment. Mothers were scored on contingent responsivity and warmth. At 12 months, the dyads underwent the Strange Situation Procedure where sensitivity was assessed using a 72‐item version of the MBQS. The findings revealed the stability of maternal sensitivity across times, contexts, and measures. However, linkages applied only to 10‐month scores from reunion, not play, episodes, supporting Pederson and Moran's (1995) assertion that high‐demand circumstances reveal more meaningful differences in maternal sensitivity. Copyright © 2012 John Wiley & Sons, Ltd.     

Domain‐General and Domain‐Specific Creative‐Thinking Tests: Effects of Gender and Item Content on Test Performance

The study examined the effects of gender and item content of domain‐general and domain‐specific creative‐thinking tests on four subscale scores of creative‐thinking (fluency, flexibility, originality, and elaboration). Chinese tenth‐grade students (234 males and 244 females) participated in the study. Domain‐general creative thinking was measured by using two domain‐independent items—box and newspaper. Domain‐specific creative thinking was measured in the domain of history by two history‐specific items—school uniform and health food—that were part of lessons in modern Chinese history. Domain‐general creative‐thinking scores were not different across gender in any of the four subscales. In domain‐specific creative thinking, female students produced more responses (fluency) and more categories of ideas (flexibility), and more detailed answers (elaboration) on both items than did males. Gender difference was not found in originality. Item effects were significant in both general and specific creative‐thinking scores, with higher fluency, flexibility, and elaboration for the newspaper than the box item, and higher fluency, flexibility, originality, and elaboration for the school uniform than the health food item. The findings on both gender and item effects support the contention that personal interest and life experience influence the generation of creative solutions. The finding that gender did not differ in domain‐general creative‐thinking was expected, as the two general items (box and newspaper) are experienced similarly by both genders. As most of the creative‐thinking tests are influenced by individuals' experience beyond creative‐thinking ability, judicial evaluation and use of creative‐thinking scores are underscored.     

Psychometric properties of the state-trait anxiety inventory for Asian/Pacific-islander adolescents

Anxiety disorders are said to be universal across all cultures and recent reviews have found relatively high prevalence rates across different countries. However, the experience and interpretation of anxiety are strongly influenced by cultural factors. Demonstrating cross-cultural equivalence of measures of anxiety is essential to assure that comparisons between cultures will result in meaningful interpretations. Despite the State-Trait Anxiety Inventory being the most researched of anxiety measures from a cross-cultural basis, there is a lack of empirical studies on the psychometric properties of the STAI with adolescent Asian/Pacific Islanders. The present study examined the STAI using a large sample of ethnically diverse high school students in Hawaii. In general, a four-factor model (State-Anxiety Absent, State-Anxiety Present, Trait-Anxiety Absent, and Trait-Anxiety Present) provided the best fit based on a series of confirmatory factor analyses. Indicators of internal consistency supported the reliability of the factors and subscales, and the inter-factor correlations reflected positively on the concurrent validity of the different STAI factor and subscale measures. This study suggested cautious use and interpretation of one particular item (Trait Item 14 = I try to avoid facing a crisis or difficulty ), and cautious application of the STAI to Filipino adolescents (particularly Filipino males). Domains for further research are discussed.     

Development of The Divorce Initiation Inventory

The status of divorce initiator versus non-initiator has demonstrated significant influence on individuals’ wellbeing throughout the divorce process. While these outcomes are relevant to both researchers and clinicians who focus on the influence of post-divorce outcomes, assessment of initiator status is generally conducted with a single question that varies across studies. The use of a single item to assess a complex, multifaceted construct such as divorce initiation poses a potential threat to the internal validity of the study. Further, single item assessments of divorce initiation are frequently inconsistent across studies, limiting the ability to synthesize the body of research as a whole. The Divorce Initiation Inventory (DII) is a five-item questionnaire that was constructed using terminology reflected in the extant literature. The purpose of this study is to describe the outcome of analyses for the development of the DII and to summarize the findings for initial assessment of reliability, criterion and construct validity. Clinical implications and directions for on-going research are discussed.     

The structure of verbal abilities in young and older adults

Four language sample measures as well as measures of vocabulary, verbal fluency, and memory span were obtained from a sample of young adults and a sample of older adults. Factor analysis was used to analyze the structure of the vocabulary, fluency, and span measures for each age group. Then an "extension" analysis was performed by using structural modeling techniques to determine how the language sample measures were related to the other measures. The measure of grammatical complexity was associated with measures of working memory including reading span and digit span. Two measures, sentence length in words and a measure of lexical diversity, were associated with the vocabulary measures. The fourth measure, propositional density, was associated with the fluency measures as a measure of processing efficiency. The structure of verbal abilities in young and older adults is somewhat different, suggesting age differences in processing efficiency.