项目表述效应对人格测验信度和效标效度的影响——以核心自我评价量表(CSES)为例

项目表述效应是指由项目表述方式的差异引起的与测量内容无关的系统变异,项目表述效应模型的统计本质是一种双因子模型。本研究以核心自我评价量表(CSES)为例,探讨项目表述效应对人格测验信效度的影响。采用核心自我评价量表、生活满意度量表和积极情感消极情感量表对340名“蚁族”进行测查。结果表明,CSES在核心自我评价特质以外,还存在一个反向题项目表述效应因子;忽视项目表述效应对CSES的同质性信度和效标关联效度有重要影响：高估CSES的同质性信度,低估核心自我评价与生活满意度、积极情感的正相关,高估核心自我评价与消极情感的负相关。     

An examination of the wording effect in the Rosenberg Self-Esteem Scale among culturally Chinese people

Previous psychometric studies of the Rosenberg Self-Esteem Scale (RSES; 1965) have shown that items with positive and negative words tend to form 2 factors instead of a single factor for global self-esteem. Recent studies using confirmatory factor analysis have indicated that there is an additional method effect behind negatively worded items. However, researchers conducted these studies using Western participants. Because J. L. Farh and B. S. Cheng (1997) suggested that culturally Chinese people tend to exhibit a modesty bias in self-evaluation, especially on positively worded items, researchers may infer that a wording effect of positively worded items would be evident for culturally Chinese people. The author examined the wording effect in the RSES for culturally Chinese people by comparing different confirmatory factor models. The author analyzed data from 2 independent samples of students at the National Taiwan University (ns = 393, 441) and a national sample of juniors recruited from 140 universities and colleges in Taiwan in 2004 (n = 28,862). Results showed that in addition to a global factor for self-esteem, method effects of positively and negatively worded items should also be specified for a model fitting culturally Chinese people.     

Cross-cultural investigation of the Questionnaire of Smoking Urges in American and Spanish smokers

This study evaluated the factor structure of the Questionnaire of Smoking Urges (QSU) across American and Spanish smokers. Using confirmatory factor analyses, the fits of one-, two-, and four-factor models of smoking craving in American and Spanish data sets were compared. The two-factor model provided the best fit in both samples. However, negatively worded items confounded the interpretation of the two-factor structure in both the American and Spanish data. The first factor had positively and negatively worded items, whereas the second had only positively worded items. The two-factor structure was reexamined, removing either the positively or the negatively worded items. Using only positively worded items resulted in a loss of fit, whereas using only negatively worded items improved model fit substantially. Thus, the results supported the generalization of smoking craving across Spanish and American smokers and suggested that negatively worded items play a larger role in the two-factor structure of the QSU than originally thought.     

The Two-Factor Solution to Allen and Meyer’s (1990) Affective Commitment Scale: Effects of Negatively Worded Items

Purpose

Affective organizational commitment is a construct important to both practitioners and researchers; thus, construct-valid measurement is needed. Allen and Meyer’s (J Occup Psychol 63:1–18, 1990) Affective Commitment Scale is a popular measure of affective organizational commitment. Although conceptualized as unidimensional, a two-factor solution sometimes emerges. Whether the two factors are substantive, reflecting AC-love and AC-joy constructs, or methodological, reflecting positively and negatively worded items, is unclear. This issue is examined in five studies.

Design/Methodology/Approach

In a new approach, conditions designed to produce cognitive fatigue are manipulated. Support for the method factor interpretation would be provided if the two-factor solution emerges only when participants are fatigued and negatively worded items are present. Cross-sample analyses are also conducted.

Findings

Analyses indicated that the two-factor solution fit the data well only when (a) participants were cognitively fatigued and (b) negatively worded items were present. This finding extended to both students and employees, and it held regardless of which items were negatively worded or whether negatively worded items were emphasized.

Implications

Results suggest that the second factor is likely methodological and seems to result from careless responding or fatigue on negatively worded items. It is suggested that users modify the items to be positively worded or administer the scale when respondents have sufficient cognitive resources to respond.

Originality/Value

This study was the first to use experimental methodology to test whether the two-factor solution is methodological or substantive. Results provide guidance for improving the scale’s construct validity in research and in practical applications.     

项目表述效应的统计控制：以中文版生活定向测验为例

本研究以中文版生活定向测验（LOT-R）为例,使用CTCM和CTCU方法进行建模,旨在探讨中文表述的量表中是否存在项目表述效应以及影响项目表述效应的特质因素。采用中文版生活定向测验等量表组成的问卷对334名大学生进行测试。结果表明,中文版LOT-R存在项目表述效应,是一种反向题的项目表述效应,也可称为语言标签效应;在分离了反向题项目表述效应后,发现中文版LOT-R只存在乐观因子,而不再有悲观因子。这表明乐观-悲观同属于一种人格特质,而非两种不同的人格特质;反向题项目表述效应是一种结构性误差,会导致量表结构的扭曲。研究还发现担心错误和父母期望与反向题项目表述效应存在显著的负相关,表明越是担心错误或父母期望水平越高的被试,他们对反向题的项目表述效应（语言标签效应）越小。社会赞许性对反向题项目表述效应没有影响。     

Confirmatory factor analysis of the Penn State Worry Questionnaire

Worry is often assessed with the 16-item Penn State Worry Questionnaire (PSWQ; Meyer, Miller, Metzger, & Borkovec, 1990. Behaviour Research and Therapy, 28, 487-495), but the factor structure of the PSWQ is not well articulated. Three factor analyses of the PSWQ have been conducted, with two (Brown, Antony, & Barlow, 1992. Behaviour Research and Therapy, 30, 33-37; van Rijsoort, Emmelkamp, & Vervaeke, 1999) retaining a unifactorial solution and one (St?ber, 1995) retaining a two-factor solution. We sought to determine the relative strengths of these solutions. Seven hundred eighty-eight undergraduates completed the PSWQ and measures of depression and anxiety. Confirmatory factor analyses indicated that the two-factor solution provided a better fit to the data. There was also evidence for a higher order worry factor. Multiple regression analyses revealed that factor 1 (Worry Engagement) explained the majority of the variance in the symptom measures. The higher order worry factor also accounted for variance in some measures. Compared to the PSWQ total score, Worry Engagement demonstrated slightly higher internal consistency and significantly stronger correlations with most measures of depression and anxiety. Factor 2 (Absence of Worry) demonstrated moderate negative correlations with Worry Engagement and the PSWQ Total Score but only modest correlations with depression and anxiety.     

人格测验中题目正反向陈述的效应

研究中将EPQ和NEO-FFI中的一半题目进行语义反转（肯定陈述的改为否定陈述,否定陈述的改为肯定陈述）,而另一半的题目陈述方向不变,形成的量表用于初测。重测时则把初测问卷中的题目全部进行语义反转。测试后得到363名大学生的正反EPQ和412名大学生的正反NEO-FFI数据。对比发现两套量表在使用正向题时信度略高,而使用反向陈述题目时效度略好;验证性因素分析发现正反向题能测量同一特质。正向题可能更易受反应偏差影响     

Mind your words: positive and negative items create method effects on the Five Facet Mindfulness Questionnaire

Mindfulness, a construct that entails moment-to-moment effort to be aware of present experiences and positive attitudinal features, has become integrated into the sciences. The Five Facet Mindfulness Questionnaire (FFMQ), one popular measure of mindfulness, exhibits different responses to positively and negatively worded items in nonmeditating groups. The current study employed confirmatory factor analysis with a large undergraduate sample to examine the validity of a hierarchical mindfulness model and whether response patterns related to item wording arose from method effects. Results indicated that a correlated facets model better explained the data and that negative and positive wording constituted substantive method effects. This study suggests that the FFMQ measures components that may relate to, but do not seem to directly reflect, a latent variable of mindfulness. The authors recommend against the use of an FFMQ total score, favoring individual scale scores, and further examination of method effects in mindfulness scales.     

Empirical validation and psychometric evaluation of the Brief Fear of Negative Evaluation Scale in patients with social anxiety disorder

The Brief Fear of Negative Evaluation Scale (BFNE; M. R. Leary, 1983a) is often used to assess fear of negative evaluation, the core feature of social anxiety disorder. However, few studies have examined its psychometric properties in large samples of socially anxious patients. Although the BFNE yields a single total score, confirmatory factor analysis indicated a 2-factor solution to be more appropriate, with the 1st factor consisting of all straightforwardly worded items (BFNE-S) and the 2nd of all reverse-scored items (BFNE-R). Support was obtained for the convergent and discriminant validity of the BFNE and BFNE-S, but not the BFNE-R. These results suggest that standard scoring of the BFNE may not be optimal for patients with social anxiety disorder.     

Worry and Pathological Worry in Patients with Psoriasis: Cross Sectional and Longitudinal Analyses of the Penn State Worry Questionnaire (PSWQ) in Four Samples of Patients

The purpose of the present study was to examine the psychometric properties of the Penn state Worry Questionnaire (PSWQ) in patients with psoriasis. A series of cross-sectional and longitudinal studies were undertaken to assess the reliability, stability and validity of the measure. Patients with psoriasis from four samples (consecutive attendees, anxious patients, depressed patients, and 6-month follow-up of consecutive attendees) completed the PSWQ and measures assessing related constructs of anxiety and depression (HADS) and coping (COPE). The clinical severity of patients psoriasis was also assessed by dermatologists using the Psoriasis Area and Severity Index (PASI). Exploratory factor analysis was undertaken and receiver operator characteristic (ROC) analyses were used to examine the clinical utility of the PSWQ cut-off score for normal and pathological worry. Exploratory factor analysis suggested that the PSWQ is essentially unidimensional in patients with psoriasis. Intraclass correlation demonstrated that, over a 6-month period, the reproducibility of the PSWQ total scale was good in patients with anxiety and moderate in patients with depression. ROC analysis suggested that the optimum cutoff differentiating pathological worry was > 60, which is commensurate with findings in adult mental health more generally. The PSWQ cutoff for pathological worry showed a lower specificity for depression than anxiety. The PSWQ is an appropriate measure of pathological worry in patients with psoriasis. The cutoff on the scale for pathological worry demonstrates appropriate conceptual overlap with subordinate distress categories of anxiety and depression.     

The reverse of social anxiety is not always the opposite: the reverse-scored items of the social interaction anxiety scale do not belong

Although well-used and empirically supported, the Social Interaction Anxiety Scale (SIAS) has a questionable factor structure and includes reverse-scored items with questionable utility. Here, using samples of undergraduates and a sample of clients with social anxiety disorder, we extend previous work that opened the question of whether the reverse-scored items belong on the scale. First, we successfully confirmed the factor structure obtained in previous samples. Second, we found the reverse-scored items to show consistently weaker relationships with a variety of comparison measures. Third, we demonstrated that removing the reverse-scored questions generally helps rather than hinders the psychometric performance of the SIAS total score. Fourth, we found that the reverse-scored items show a strong relationship with the normal personality characteristic of extraversion, suggesting that the reverse-scored items may primarily assess extraversion. Given the above results, we suggest investigators consider performing data analyses using only the straightforwardly worded items of the SIAS.     

The factor structure and screening utility of the Social Interaction Anxiety Scale

The widely used Social Interaction Anxiety Scale (SIAS; R. P. Mattick & J. C. Clarke, 1998) possesses favorable psychometric properties, but questions remain concerning its factor structure and item properties. Analyses included 445 people with social anxiety disorder and 1,689 undergraduates. Simple unifactorial models fit poorly, and models that accounted for differences due to item wording (i.e., reverse scoring) provided superior fit. It was further found that clients and undergraduates approached some items differently, and the SIAS may be somewhat overly conservative in selecting analogue participants from an undergraduate sample. Overall, this study provides support for the excellent properties of the SIAS's straightforwardly worded items, although questions remain regarding its reverse-scored items.     

Assessing worry in older adults: confirmatory factor analysis of the Penn State Worry Questionnaire and psychometric properties of an abbreviated model

The assessment of worry among older adults typically has involved measures designed with younger cohorts. Because of special concerns in assessing older adults, modifications to existing instruments may be necessary. Addressing equivocal factor analytic data on the Penn State Worry Questionnaire (PSWQ) among younger adults, the authors conducted confirmatory factor analyses to evaluate the generalizability of previous models to older adults with generalized anxiety disorder. Data fit poorly with established single- and two-factor models. The single-factor model was modified, resulting in the elimination of 8 items, strong fit indices, high internal consistency, adequate test-retest reliability, and good convergent and divergent validity. Further psychometric work is required to assess whether the revised model is a more parsimonious method to assess late-life anxiety.     

Confirmatory factor analysis of the Penn State Worry Questionnaire: Multiple factors or method effects?

The latent structure of the Penn State Worry Questionnaire (PSWQ) was evaluated with confirmatory factor analyses (CFAs) in 1200 outpatients with DSM-IV anxiety and mood disorders. Of particular interest was the comparative fit and interpretability of a two-factor solution (cf. Behaviour Research and Therapy 40 (2002) 313) vs. a one-factor model that specified method effects arising from five reverse-worded items. Consistent with prediction, the superiority of the one-factor model was demonstrated in split-sample CFA replications (ns=600). Multiple-group CFAs indicated that the measurement properties of the PSWQ were invariant in male and female patients. In addition to their direct relevance to the psychometrics of the PSWQ, the results are discussed in regard to methodological considerations for using factor analytic methods in the evaluation of psychological tests.     

Validation of Basic Empathy Scale: Exploring a Korean Version

The Basic Empathy Scale was designed to measure the multidimensional aspects of empathetic feelings, such as cognitive and affective empathy. The purpose of this study was to evaluate the reliability and validity of the Korean version of the Basic Empathy Scale (BES). The Korean version of BES was administered to a sample of early adolescents aged 12–14 (N = 1524). The result of confirmatory factor analysis showed a two-factor solution with method effects associated with negatively worded items to be the most adequate model. Empathy was positively correlated with the measure of attachment, negatively correlated with the measure of bullying problems, and exhibited satisfactory internal consistency. BES proved to be a reliable assessment tool for empathy in a Korean adolescent sample. Moreover, in line with the previous research, girls were found to score significantly higher in both cognitive and affective empathy than boys.     

Differential item functioning on the Five Facet Mindfulness Questionnaire is minimal in demographically matched meditators and nonmeditators

A recent study of the Five Facet Mindfulness Questionnaire reported high levels of differential item functioning (DIF) for 18 of its 39 items in meditating and nonmeditating samples that were not demographically matched. In particular, meditators were more likely to endorse positively worded items whereas nonmeditators were more likely to deny negatively worded (reverse-scored) items. The present study replicated these analyses in demographically matched samples of meditators and nonmeditators (n = 115 each) and found that evidence for DIF was minimal. There was little or no evidence for differential relationships between positively and negatively worded items for meditators and nonmeditators. Findings suggest that DIF based on items' scoring direction is not problematic when the Five Facet Mindfulness Questionnaire is used to compare demographically similar meditators and nonmeditators.     

Global Self-Esteem as a Correlate of Work-Related Attitudes: A Question of Dimensionality

We examined the dimensionality of one of the most frequently used measures of global self-esteem, the Rosenberg Self-Esteem Scale (RSE; Rosenberg, 1965). Although the RSE is assumed to be unidimensional, Kaplan and Pokorny (1969) provided evidence that the scale taps two independent constructs: self-derogation versus defensive assertion of self-worth, in our study a confirmatory factor analysis was conducted which supported the superiority of the Kaplan and Pokorny two-factor model over a one-factor model but indicated that their model provided a poor fit to the data. Consequently, an exploratory factor analysis was conducted which revealed two dimensions, one mostly defined by negatively worded items (self-derogation) and the second by positively worded items (self-enhancement). Although the correlations obtained were generally quite small, relations with work-related attitudes shown for the negative factor were in the opposite direction and somewhat larger than those shown for the positive factor. Thus, additional support was provided for a two-dimensional conceptualization of the scales.     

Measuring the Intrapersonal Component of Psychological Empowerment: Confirmatory Factor Analysis of the Sociopolitical Control Scale

The Sociopolitical Control Scale (SPCS) is a widely used measure of the intrapersonal component of psychological empowerment. Confirmatory factor analyses (CFA) were conducted with data from two samples to test the hypothesized structure of the SPCS, the potential effects of method bias on the measure’s psychometric properties, and whether a revised version of the scale (SPCS-R) yielded improved model fit. Sample 1 included 316 randomly selected community residents of the Midwestern United States. Sample 2 included 750 community residents of the Northeastern U.S. Results indicated that method bias from the use of negatively worded items had a significant effect on the factor structure of the SPCS. CFA of the SPCS-R, in which negatively worded items were rephrased so that all statements were positively worded, supported the measure’s hypothesized two-factor structure (i.e., leadership competence and policy control). Subscales of the SPCS-R were found reliable and related in expected ways with measures of community involvement. Implications of the study for empowerment-based research and practice are described, and strategies to further develop the SPCS are discussed.     

Structured personality test item characteristics and validity

Using the structured personality test item as the unit of analysis, the purpose of this research was to evaluate the relationship between validity and a variety of other test item parameters. Of particular interest was the relationship of test item criterion validity to negative keying and to negative wording. By drawing a distinction between negative keying and negative wording it was demonstrated that the use of balanced scales to control acquiescence need not result in a reduction in item criterion validity. Whereas the use of negative wording has in the past reduced validity, data demonstrated that positively worded, negatively keyed items did not. In addition, results indicated that clear, moderately short, relevant test items tended to be the most empirically valid.     

随机截距因子分析模型在控制条目表述效应中的应用

本研究用中文修订版罗森博格自尊量表(RSES-R)考察随机截距因子分析模型在控制条目表述效应时的表现。用RSES-R和过分宣称问卷组成的量表调查621名中学生。结果表明,随机截距模型在建模时,拟合指数良好、因子方差与负荷合理,自尊因子分与RSES-R总分有极高相关,表明该模型能有效分离RSES-R得分的特质与表述效应。分离的表述效应因子分与受测者的自我提升水平具有显著但较弱的相关,表明表述效应与自受测者的社会赞许性有共同的成分。