The effect of varying the number of response alternatives in rating scales: Experimental evidence from intra-individual effects

Despite a hundred years of questionnaire testing, no consensus has been reached on the optimal number of response alternatives in rating scales. Differences in prior research may have been due to the use of various psychometric models (classical test theory, item factor analysis, and item response theory) and different performance criteria (reliability, convergent/discriminant validity, and internal structure of the questionnaire). Furthermore, previous empirical studies on this issue have tackled the experimental design from a between-subjects perspective, thus ignoring intra-individual effects. In contrast with this approach, we propose a within-subjects experimental design and a comprehensive statistical methodology using structural equation models for studying all of these aspects simultaneously, therefore increasing statistical power. To illustrate the method, two personality questionnaires were examined using a repeated measures design. Results indicated that as the number of response alternatives increased, (1) internal consistency increased, (2) there was no effect on convergent validity, and (3) goodness of fit worsened. Finally, the article assesses the practical consequences of this research for the design of future personality questionnaires.     

On ad hoc personality scales: A reply to Burisch

Two recent articles by M. Burisch (1984, American Psychologist, 37, 214–227; 1984, Journal of Research in Personality, 18, 81–98) have presented data bearing on the traditional methods by which personality is assessed. Based on a review of selected empirical studies and analyses of new data, Burisch has concluded that (a) single-item global self-ratings generally yield greater coefficients of criterion validity than questionnaire measures of the same traits, and (b) most existing personality questionnaires can be shortened considerably without any consequent loss in validity. These conclusions stand in direct contrast to previous appeals by Epstein and others for the use of extensive aggregate measures of personality derived according to the axioms of psychometric theory. In response to Burisch, we describe measurement artifacts and strategies of test item selection that account for some of his conclusions and we reiterate the need for the application of conventional construct-based questionnaires in the assessment of personality.     

Psychometric and Validity Issues in Machine Learning Approaches to Personality Assessment: A Focus on Social Media Text Mining

In the age of big data, substantial research is now moving toward using digital footprints like social media text data to assess personality. Nevertheless, there are concerns and questions regarding the psychometric and validity evidence of such approaches. We seek to address this issue by focusing on social media text data and (i) conducting a review of psychometric validation efforts in social media text mining (SMTM) for personality assessment and discussing additional work that needs to be done; (ii) considering additional validity issues from the standpoint of reference (i.e. ‘ground truth’) and causality (i.e. how personality determines variations in scores derived from SMTM); and (iii) discussing the unique issues of generalizability when validating SMTM for personality assessment across different social media platforms and populations. In doing so, we explicate the key validity and validation issues that need to be considered as a field to advance SMTM for personality assessment, and, more generally, machine learning personality assessment methods. © 2020 European Association of Personality Psychology     

Examining the item response process to personality measures in high-stakes situations: Issues of measurement validity and predictive validity

We conducted two experimental studies with between-subjects and within-subjects designs to investigate the item response process for personality measures administered in high- versus low-stakes situations. Apart from assessing measurement validity of the item response process, we examined predictive validity; that is, whether or not different response models entail differential selection outcomes. We found that ideal point response models fit slightly better than dominance response models across high- versus low-stakes situations in both studies. Additionally, fitting ideal point models to the data led to fewer items displaying differential item functioning compared to fitting dominance models. We also identified several items that functioned as intermediate items in both the faking and honest conditions when ideal point models were fitted, suggesting that ideal point model is “theoretically” more suitable across these contexts for personality inventories. However, the use of different response models (dominance vs. ideal point) did not have any substantial impact on the validity of personality measures in high-stakes situations, or the effectiveness of selection decisions such as mean performance or percent of fakers selected. These findings are significant in that although prior research supports the importance and use of ideal point models for measuring personality, we find that in the case of personality faking, though ideal point models seem to have slightly better measurement validity, the use of dominance models may be adequate with no loss to predictive validity.     

The validity of formal and informal personality assessments

Two recent articles by M. Burisch (1984, American Psychologist, 37, 214–227; 1984, Journal of Research in Personality, 18, 81–98) have presented data bearing on the traditional methods by which personality is assessed. Based on a review of selected empirical studies and analyses of new data, Burisch has concluded that (a) single-item global self-ratings generally yield greater coefficients of criterion validity than questionnaire measures of the same traits, and (b) most existing personality questionnaires can be shortened considerably without any consequent loss in validity. These conclusions stand in direct contrast to previous appeals by Epstein and others for the use of extensive aggregate measures of personality derived according to the axioms of psychometric theory. In response to Burisch, we describe measurement artifacts and strategies of test item selection that account for some of his conclusions and we reiterate the need for the application of conventional construct-based questionnaires in the assessment of personality.     

Personality Factors S,E, and M,and their Measurement

Terms such as often, usually, and sometimes frequently modify items found in personality questionnaires. Semantic style refers to individual differences in the quantitative meanings subjects give such terms. This study investigated the relation between semantic style variables and questionnaire measures of social desirability, extraversion, and neuroticism. Significant associations were found for two of these three personality scales: social desirability and neuroticism. If semantic style is shown to be pervasive and appreciable in its influence on questionnaire response, its systematic control should become a psychometric goal of some importance. Techniques are discussed for thus achieving more valid questionnaire measures.     

Measuring the validity and psychometric properties of a short form of the Hypomanic Personality Scale

The Hypomanic Personality Scale (HPS) is used to investigate hypomanic traits and risk for bipolar spectrum disorders; however, the length of the HPS (48 items) may be prohibitive for clinical research and screening purposes. Meads and Bentall (2008) developed a promising 20-item version of the HPS; however, the psychometric properties and validity of the short form have not been thoroughly examined. The present study investigated the construct validity and psychometric properties of the short HPS. A sample of 2713 non-clinically ascertained young adults was used to assess psychometric properties of the short form relative to the original scale. Two non-overlapping subsamples (n = 522; n = 145) were used to investigate the validity of the short HPS using personality and temperament questionnaires and clinical interviews of bipolar psychopathology and diagnoses. The short and original HPS generally had comparable correlations with measures of temperament, personality, impulsivity, borderline personality, grandiosity, psychosocial functioning, and alcohol use, and comparably predicted DSM bipolar and bipolar spectrum diagnoses. Overall, the short HPS was found to be both reliable and valid. However, the short HPS tended to be more strongly correlated with pathological components of hypomanic personality and less strongly correlated with exuberant and potentially adaptive aspects of the construct.     

Nonverbal assessment of the Big Five personality factors

The Nonverbal Personality Questionnaire (NPQ) is an experimental, structured, nonverbal measure of 16 personality traits. Its items lack verbal content and, therefore, the inventory is useful for cross‐cultural research. Our goal is this research was to select a subset of the NPQ items to form a new nonverbal questionnaire based on the Five‐Factor Model of personality. We describe the construction of the Five‐Factor Nonverbal Personality Questionnaire (FF‐NPQ), and present data on its psychometric properties. These data include scale internal consistencies, intercorrelations, convergences with verbal measures of the Big Five factors, discriminant validity correlations, correlations with peer ratings, and ability to predict socially important behaviour criteria such as smoking and alcohol consumption. In a second study, we report on the psychometric properties of the FF‐NPQ in an independent sample of respondents from seven different countries. The utility of the new nonverbal inventory for cross‐cultural research is discussed. Copyright © 2001 John Wiley & Sons, Ltd.     

A Psychometric Review of the Personality Inventory for DSM–5 (PID–5): Current Status and Future Directions

The paradigm of personality psychopathology is shifting from one that is purely categorical in nature to one grounded in dimensional individual differences. Section III (Emerging Measures and Models) of the Diagnostic and Statistical Manual of Mental Disorders (5th ed. [DSM–5]; American Psychiatric Association, 2013), for example, includes a hybrid categorical/dimensional model of personality disorder classification. To inform the hybrid model, the DSM–5 Personality and Personality Disorders Work Group developed a self-report instrument to assess pathological personality traits—the Personality Inventory for the DSM–5 (PID–5). Since its recent introduction, 30 papers (39 samples) have been published examining various aspects of its psychometric properties. In this article, we review the psychometric characteristics of the PID–5 using the Standards for Educational and Psychological Testing as our framework. The PID–5 demonstrates adequate psychometric properties, including a replicable factor structure, convergence with existing personality instruments, and expected associations with broadly conceptualized clinical constructs. More research is needed with specific consideration to clinical utility, additional forms of reliability and validity, relations with psychopathological personality traits using clinical samples, alternative methods of criterion validation, effective employment of cut scores, and the inclusion of validity scales to propel this movement forward.     

Assessment of the behavioral inhibition system and the behavioral approach system: adaptation and validation of the Sensitivity to Punishment and Sensitivity to Reward Questionnaire (SPSRQ) in a Chilean sample

The goal of the present study is to estimate the psychometric properties of the Sensitivity to Punishment and Sensitivity to Reward Questionnaire (SPSRQ; Torrubia, Avila, Moltó, & Caseras, 2001) in a sample of Chilean college students. The main hypothesis is that the instrument would show appropriate levels of reliability and validity, in light of previous validation studies. A pilot study was conducted in order to generate the adapted version of the questionnaire, which was then applied to a student sample from different undergraduate careers (n = 434). The results show the expected levels of reliability (test-retest and internal consistency). The factorial validity does not comply with the expected model, suggesting a further consideration of the structure of the questionnaire. External validity is appropriate, as the questionnaire shows the expected correlations with other personality measures. It is concluded that the SPSRQ is adequate for the context of validation, and this study contributes to the generalization of the questionnaire, since the results are consistent with the expected psychometric properties that have been reported in the literature.     

心理健康素质测评系统·中国成年人认知风格问卷的编制

通过对认知风格既往文献的回顾和分析,从临床心理学的角度将认知风格定义为个体在解释生活事件时所偏爱的和习惯化的特征性认知方式,并确定了4个维度：过度概括化、选择性提取、个体化和灾难化。通过开放式问卷调查、参考国内外已有量表、自编项目及专家评定,形成27个项目的认知风格问卷初测版,并分别在大学生样本和非学生样本中进行测试,筛选出在两个样本中都符合心理测量学要求的16个项目,最终形成认知风格问卷正式版。对认知风格问卷正式版进行心理测量学指标考察,结果发现问卷的内部一致性信度、重测信度、结构效度、内容效度、聚合效     

Factor analysis of the Spanish Burnout Inventory among public administration employees

Burnout has been recognized as an important stress‐related problem. Researchers have been troubled by some of the psychometric limitations of the questionnaires developed to evaluate burnout. This study was designed to assess the factor structure of the Spanish Burnout Inventory in a sample of 548 Brazilian public administration employees. This instrument comprises 20 items distributed in four dimensions: enthusiasm toward the job (5 items), psychological exhaustion (4 items), indolence (6 items), and guilt (5 items). The factor structure was examined through confirmatory factor analysis. To assess the factorial validity of the Spanish Burnout Inventory, four alternative models were tested. The four‐factor model obtained an adequate data fit for the sample, χ²(164) = 514.358, p < .001, RMSEA = .062, GFI = .910, NFI = .915, CFI = .940, and AIC = 606.358. The results showed that the four‐model factor of the Spanish Burnout Inventory possesses adequate psychometric properties in the Brazilian cultural context. Implications for future research and practice are also discussed.     

Psychometric Properties of Parent–Child (0–5 years) Interaction Outcome Measures as Used in Randomized Controlled Trials of Parent Programs: A Systematic Review

This systematic review sought to identify observational measures of parent–child interactions commonly implemented in parenting program research, and to assess the level of psychometric evidence available for their use with this age group. Two separate searches of the same databases were conducted; firstly, to identify eligible instruments, and secondly to identify studies reporting on the psychometric properties of the identified measures. Five commercial platforms hosting 19 electronic databases were searched from their inception to conducted search dates. Fourteen measures were identified from Search 1; a systematic search of randomized controlled trial evaluations of parenting programs. For Search 2, inclusion/exclusion criteria were applied to 1327 retrieved papers that described the development and/or validation of the 14 measures identified in Search 1. Seventeen articles met the inclusion criteria, resulting in five observational measures for the final review. Data were extracted and synthesized using the COSMIN rating system to describe the methodological quality of each article alongside the overall quality rating of the psychometric property reported for each measure using the Terwee checklist. Measure reliability was categorized into four domains (internal consistency, test-re-test, inter-rater, and intra-rater). Measure validity was categorized into four domains (content, structural, convergent/divergent, and discriminant). Results indicated that the majority of psychometric evidence related to children aged from birth the three with internal consistency, inter-rater reliability, and structural validity the most commonly reported properties, although this evidence was often weak. The findings suggest further validation of the included measures is required to establish acceptability for the whole target age group.

     

Trait-specific dependence in romantic relationships

ABSTRACT Informed by three theoretical frameworks—trait psychology, evolutionary psychology, and interdependence theory—we report four investigations designed to develop and test the reliability and validity of a new construct and accompanying multiscale inventory, the Trait-Specific Dependence Inventory (TSDI). The TSDI assesses comparisons between present and alternative romantic partners on major dimensions of mate value. In Study 1, principal components analyses revealed that the provisional pool of theory-generated TSDI items were represented by six factors: Agreeable/Committed, Resource Accruing Potential, Physical Prowess, Emotional Stability, Surgency, and Physical Attractiveness. In Study 2, confirmatory factor analysis replicated these results on a different sample and tested how well different structural models fit the data. Study 3 provided evidence for the convergent and discriminant validity of the six TSDI scales by correlating each one with a matched personality trait scale that did not explicitly incorporate comparisons between partners. Study 4 provided further validation evidence, revealing that the six TSDI scales successfully predicted three relationship outcome measures—love, time investment, and anger/upset—above and beyond matched sets of traditional personality trait measures. These results suggest that the TSDI is a reliable, valid, and unique construct that represents a new trait-specific method of assessing dependence in romantic relationships. The construct of trait-specific dependence is introduced and linked with other theories of mate value.     

Structural equation models and personality research

This article is a nontechnical introduction to the use of structural equation models in personality research Although such models can be fruitfully used to address a variety of important theoretical issues, the substantive focus in this article is on the use of such models for elucidating the construct validity of personality measures We include numerous more specific topics under our treatment of construct validity First of all, we show how structural equation models can be applied to the issues of convergent and discriminant validity Do our variables measure the constructs we want them to measure and not other constructs that we would prefer not to measure? Second, we show the utility of structural equation models for predictive validity Do our variables reliably predict other constructs with which they are theoretically linked? Finally, we examine the stability of personality constructs through structural equation models Through-out, our emphasis is on the particular advantages that structural equation models bring to these analytic tasks Ultimately, such models must be used in the service of theory, and when used appropriately, they can help us to refine both our measures and our theories of individual differences     

The Short Dark Triad (SD3): Adaptation and Psychometrics among At-Risk Male and Female Youths

The dark triad is a term used to describe a constellation of three socially harmful personality traits: Machiavellianism, psychopathy, and narcissism. The main aim of the present study was to examine the psychometric properties of the Short Dark Triad (SD3) scale, a brief measure of the dark triad, among a sample (N = 412) of male (n = 200) and female (n = 212) at-risk Portuguese youths. Confirmatory factor analysis revealed the three-factor structure of the SD3 obtained an adequate fit, and measurement invariance across gender was demonstrated, although two items per dimension had to be removed. The Portuguese version of the SD3 scale demonstrated satisfactory psychometric properties, specifically the properties of internal consistency, convergent validity, discriminant validity, and known-groups validity, which overall justifies its use among this population. Findings are discussed in terms of the use of the SD3 scale with male and female at-risk youths.     

Development of the Treatment Acceptability Questionnaire

This article describes the development and psychometric properties of a measure designed to assess the treatment acceptability of psychological treatments for both adult and child populations. Initial evidence on the Treatment Acceptability Questionnaire (TAQ) indicates that it has a good internal consistency and test-retest stability (over a 3-week period). The pattern of correlations with other measures of treatment acceptability for child-oriented interventions provides evidence of concurrent validity. Suggestions are made regarding the further validation and use of the TAQ.     

A Serbian Version of Modified and Revised Experiences in Close Relationships Scale (SM–ECR–R)

We present details about the validation of the Serbian version of the Experiences in Close Relationships–Revised questionnaire modified to measure attachment in close relationships in general (SM–ECR–R). Its psychometric features were examined with 2 samples of students (N = 719 and N = 91) and 1 group of employed adults (N = 259). The results obtained in the student sample were equivalent to those obtained in validation studies worldwide, whereas in the more heterogeneous sample of employed adults, internal consistency of both avoidance and anxiety was lower and their intercorrelation was higher. Test–retest reliability was satisfactory, and relations with measures of attachment, empathy, and personality structure were meaningful. The results support the SM–ECR–R as a reliable and valid research instrument, but its internal structure and consistency might vary in different samples, depending on age, education, gender, and culture.     

Validation française d’une échelle d’anxiété spécifique à la grossesse (PRAQ-R2)

ObjectivesPregnancy specific anxiety (PSA) may have many consequences on the health of the unborn child and the mother. Nowadays, in French, there is no validated tool to measure it. The aim of this study is to validate the French version of the Pregnancy Related Anxiety Questionnaire (PRAQ-R2). This 10 items scale measures three PSA dimensions: worries about child health, fear of giving birth and concern about own appearance.MethodsAfter a translation and a back-translation step, the structural validity and the internal consistency of the scale were assessed in 160 pregnant women recruited online. Convergent validity (STAI-Y) and correlates of the construct were also examined.ResultsThe model fit indices are satisfactory, confirming the three- factor structure of the PRAQ-R2 in its French version. The dimensions and the total scores have a good internal consistency (α > .80), and convergent validity was also demonstrated.ConclusionThe PRAQ-R2 is the first tool designed to measure PSA and validated in France. This French validation has promising psychometric qualities. However, its discriminant validity and stability remain to be explored.