Being inconsistent about consistency: when coefficient alpha does and doesn't matter

One of the central tenets of classical test theory is that scales should have a high degree of internal consistency, as evidenced by Cronbach's a, the mean interitem correlation, and a strong first component. However, there are many instances in which this rule does not apply. Following Bollen and Lennox (1991), I differentiate between questionnaires such as anxiety or depression inventories, which are composed of items that are manifestations of an underlying hypothetical construct (i.e., where the items are called effect indicators) and those such as Scale 6 of the Minnesota Multiphasic Personality Inventory (Hathaway & McKinley, 1943) and ones used to tap quality of life or activities of daily living in which the items or subscales themselves define the construct (these items are called causal indicators). Questionnaires of the first sort, which are referred to as scales in this article, meet the criteria of classical test theory, whereas the second type, which are called indexes here, do not. I discuss the implications of this difference for how items are selected, the relationship among the items, and the statistics that should and should not be used in establishing the reliability of the scale or index.     

Dimensionality and construct validity of a video-based, objective personality test for the assessment of willingness to take risks in road traffic

The study at hand reports first results about the dimensionality and construct validity of a newly developed objective, video-based personality test, which assesses the willingness to take risks in traffic situations. On the basis of the theory of risk homeostasis developed by Wilde, different traffic situations with varying amounts of objective danger were filmed. These situations mainly consisted of situations with passing maneuvers and speed choice or traffic situations at intersections. Each of these traffic situations describes an action which should be carried out. The videos of the traffic situations are presented twice. Before the first presentation, a short written explanation of the preceding traffic situation and a situation-contingent reaction is provided. The respondents are allowed to obtain an overview of the given situations during the first presentation of each traffic situation. During the second presentation the respondents are asked to indicate at which point the action that is contingent on the described situation will become too dangerous to carry out. Latencies for items were recorded as a measure for the magnitude of the person's subjectively accepted willingness to take risks in the sense of the risk homeostasis theory by Wilde. In a study with 243 people with different education and sex, the one-dimensionality of the test corresponding to the latency model by Scheiblechner was investigated. Analysis indicated that the new measure assesses a one-dimensional latent personality trait which can be interpreted as subjectively accepted amount of risk (target risk value). First indicators for the construct validity of the test are given by a significant correlation with the construct-related secondary scale, adventurousness of the Eysenck Personality Profiler with, at the same time, nonsignificant correlations to the two secondary scales, extroversion and emotional stability, that are not linked to the construct.     

Psychometric Properties of the Wisconsin Schizotypy Scales in an Undergraduate Sample: Classical Test Theory,Item Response Theory,and Differential Item Functioning

The Wisconsin Schizotypy Scales are widely used for assessing schizotypy in nonclinical and clinical samples. However, they were developed using classical test theory (CTT) and have not had their psychometric properties examined with more sophisticated measurement models. The present study employed item response theory (IRT) as well as traditional CTT to examine psychometric properties of four of the schizotypy scales on the item and scale level, using a large sample of undergraduate students (n = 6,137). In addition, we investigated differential item functioning (DIF) for sex and ethnicity. The analyses revealed many strengths of the four scales, but some items had low discrimination values and many items had high DIF. The results offer useful guidance for applied users and for future development of these scales.     

Knowledge as a formative construct: A good alpha is not always better

Measurements of domain knowledge very often use and report Cronbach's alpha or similar indicators of internal consistency for test construction. In this short article, we argue that this approach is often at odds with the theoretical conception of knowledge underlying the measure. While domain knowledge is usually described as a formative construct (formed by the manifest observations) theoretically, the use of Cronbach's alpha to construct and evaluate an empirical measure implies a reflective model (the construct reflects in manifest behaviors). After illustrating the difference between reflective and formative models, we illustrate how this mismatch between theoretical conception and empirical operationalization can have substantial implications for the assessment and modeling of domain knowledge. Specifically, the construct may be operationalized too narrowly or even be misinterpreted by applying criteria for item selection that focus on homogeneity such as Cronbach's alpha. Rather than maximizing items internal consistency, researchers constructing measures of domain knowledge should, therefore, make strong arguments for the theoretical merit of their items even if they are not correlated to each other.     

Constructing personality scales under the assumptions of an ideal point response process: toward increasing the flexibility of personality measures

The main aim of this article is to explicate why a transition to ideal point methods of scale construction is needed to advance the field of personality assessment. The study empirically demonstrated the substantive benefits of ideal point methodology as compared with the dominance framework underlying traditional methods of scale construction. Specifically, using a large, heterogeneous pool of order items, the authors constructed scales using traditional classical test theory, dominance item response theory (IRT), and ideal point IRT methods. The merits of each method were examined in terms of item pool utilization, model-data fit, measurement precision, and construct and criterion-related validity. Results show that adoption of the ideal point approach provided a more flexible platform for creating future personality measures, and this transition did not adversely affect the validity of personality test scores.     

Burdened Societies and Transitional Justice

Following John Rawls, nonideal theory is typically divided into: (1) “partial-compliance theory” and (2) “transitional theory." The former is concerned with those circumstances in which individuals and political regimes do not fully comply with the requirements of justice, such as when people break the law or some individuals do not do their fair share within a distributive scheme. The latter is concerned with circumstances in which background institutions may be unjust or may not exist at all. This paper focuses on issues arising in transitional theory. In particular, I am concerned with what Rawls’ has called “burdened societies," that is, those societies that find themselves in unfavorable conditions, such that their historical, social or economic circumstances make it difficult to establish just institutions. The paper investigates exactly how such burdened societies should proceed towards a more just condition in an acceptable fashion. Rawls himself tells us very little, except to suggest that societies in this condition should look for policies and courses of action that are morally permissible, politically possible and likely to be effective. In this paper I first try to anticipate what a Rawlsian might say about the best way for burdened societies to handle transitional problems and so move towards the ideal of justice. Next, I construct a model of transitional justice for burdened societies. Ultimately, I argue for a model of transitional justice that makes use of a nonideal version of Rawls’ notion of the worst-off representative person.     

Hardiness: a review of theory and research.

Although a large body of research on hardiness (a personality construct with dimensions of commitment, control, and challenge) has accumulated, several fundamental issues remain unresolved. Although there are several hardiness scales, the properties of these scales have not been compared. There is debate as to whether hardiness is one or several characteristics. Research studying the pathways through which hardiness exerts its effects has not been comprehensively evaluated. Whereas critics have argued that hardiness does not buffer stress, others have suggested that hardiness buffers for working adults, for males, and in prospective analyses. There is also growing concern that hardiness is related to neuroticism. A review of the literature supports the following conclusions: The Dispositional Resilience Scale (DRS) has several advantages over alternative scales; DRS items form three factors that are consistent with hardiness theory; hardiness dimensions generally show low to moderate intercorrelations; the most common way of categorizing subjects as high or low in hardiness is not consistent with hardiness theory; hardiness does not buffer stress, and it does not buffer stress for working adults, for males, or in prospective analyses; both old and new hardiness scales inadvertently measure neuroticism. Recommendations for future research are provided.     

A latent-trait based reliability estimate and upper bound

An estimate and an upper-bound estimate for the reliability of a test composed of binary items is derived from the multidimensional latent trait theory proposed by Bock and Aitkin (1981). The estimate derived here is similar to internal consistency estimates (such as coefficient alpha) in that it is a function of the correlations among test items; however, it is not a lowerbound estimate as are all other similar methods.An upper bound to reliability that is less than unity does not exist in the context of classical test theory. The richer theoretical background provided by Bock and Aitkin's latent trait model has allowed the development of an index (called here) that is always greater-than or equal-to the reliability coefficient for a test (and is less-than or equal-to one). The upper bound estimate of reliability has practical uses—one of which makes use of the greatest lower bound.     

Towards More Rigorous Personality Trait–Outcome Research

Much of personality research attempts to identify causal links between personality traits and various types of outcomes. I argue that causal interpretations require traits to be seen as existentially and holistically real and the associations to be independent of specific ways of operationalizing the traits. Among other things, this means that, to the extents that causality is to be ascribed to such holistic traits, items and facets of those traits should be similarly associated with specific outcomes, except for variability in the degrees to which they reflect the traits (i.e. factor loadings). I argue that, before drawing causal inferences about personality trait–outcome associations, the presence of this condition should be routinely tested by, for example, systematically comparing the outcome associations of individual items or facets, or sampling different indicators for measuring the same purported traits. Existing evidence suggests that observed associations between personality traits and outcomes at least sometimes depend on which particular items or facets have been included in trait operationalizations, calling trait‐level causal interpretations into question. However, this has rarely been considered in the literature. I argue that when outcome associations are specific to facets, they should not be generalized to traits. Furthermore, when the associations are specific to particular items, they should not even be generalized to facets. Copyright © 2016 European Association of Personality Psychology     

An examination of the MMPI-2 Wiener-Harmon Subtle subscales for D and Hy: implications for parent scale and neurotic triad interpretation

This research explores the scale development process for the MMPI-2 Wiener and Harmon (1946) Subtle subscales for Depression (D) and Hysteria (Hy) to provide insight into why certain items were included on these scales and were subsequently but inappropriately assumed to be subtle indicators of the same pathology that the Obvious items measure. In this research, I also explore what the Subtle scales on D and Hy measure and their potential utility for the interpretation of their parent scales and the "neurotic triad." It was hypothesized that the D and Hy Subtle subscales are related to denial, repression, or both and this hypothesis was supported. In a sample of 1,240 inpatient and outpatient psychiatric patients at a large Army medical center, it was found that these subscales had strong positive correlations with othe scales on the Minnesota Multiphasic Personality Inventory-2 (MMPI-2; Butcher, Dahlstrom. Graham, Tellegen, & Kaemmer, 1989) related to denial, repression, or both. It was also found that they had strong negative correlations with scales on the MMPI-2 and Millon Clinical Multiaxial Inventory (MCMI-II; Millon, 1987) that are related to symptom endorsement, which can be considered the opposite of denial or repression. In addition, ratings of the Subtle items on D and Hy by clinical psychology residents were consistent with the hypothesis that these items reflect a denial of psychological or physical dysfunction.     

Measurement of agency, communion, and emotional vulnerability with the Personal Attributes Questionnaire

Factor analytic studies of the 24-item Personal Attributes Questionnaire (Spence & Helmreich, 1978) have reported inconsistent results, and a previous confirmatory factor analysis (CFA) indicated inadequate fit for factors corresponding to Masculinity, Femininity, and Masculinity-Femininity scales. In this research, we used CFA in a college sample (N = 382) to evaluate the 3-factor model, and we revised scales by eliminating 6 misspecified items. The revised model fit well in another college sample (N = 230). We renamed the revised scales Agency, Communion, and Emotional Vulnerability. In relation to Five-factor theory, Emotional Vulnerability and Communion correlated well with Neuroticism and Agreeableness, respectively, and Agency had moderate correlations with Neuroticism, Extraversion, and Conscientiousness. Psychometric results in the context of current theory suggest that Agency (Masculinity) may not be a fully adequate measure of the agency construct.     

Substance or style? An investigation of the NEO-PI-R validity scales

The Revised NEO Personality Inventory (NEO-PI-R; Costa & McCrae, 1992b) has been criticized for the absence of validity scales designed to detect response distortion. Recently, validity scales were developed from the items of the NEO-PI-R (Schinka, Kinder, & Kremer, 1997) and several studies have used a variety of methods to test their use. However, it is controversial whether these scales are measuring something that is substantive (such as psychopathology or its absence) or stylistic (which might be effortful distortion or less conscious processes such as lack of insight). In this study, we used a multimethod-multitrait approach to examine the validity of these scales in a clinical sample of 668 participants diagnosed with personality disorders or major depression. Using various indicators of both stylistic and substantive variance, confirmatory factor analyses (CFA) suggested that these validity scales measure something that may be conceptually distinct from, yet highly related to, substantive variance in responding.     

问卷数据建模的前传

问卷法是一种常见的实证研究方法。问卷数据建模的前期工作,就像是一栋大楼的奠基工程,基础是否扎实,影响后续的工程质量。本文专门讨论统计模型建立之前要做的事情（重点是量表评价）,内容包括：处理缺失值、评价量表的结构效度和题目删除的适当性、多维量表需要合成总分时检验同质性并计算合成信度、检验共同方法偏差和评价（变量）区分效度、题目打包、检验自变量的多重共线性,最后也涉及建模理据和无关变量控制等。     

Mereological nihilism: quantum atomism and the impossibility of material constitution

Mereological nihilism is the philosophical position that there are no items that have parts. If there are no items with parts then the only items that exist are partless fundamental particles, such as the true atoms (also called philosophical atoms) theorized to exist by some ancient philosophers, some contemporary physicists, and some contemporary philosophers. With several novel arguments I show that mereological nihilism is the correct theory of reality. I will also discuss strong similarities that mereological nihilism has with empirical results in quantum physics. And I will discuss how mereological nihilism vindicates a few other theories, such as a very specific theory of philosophical atomism, which I will call quantum abstract atomism. I will show that mereological nihilism also is an interpretation of quantum mechanics that avoids the problems of other interpretations, such as the widely known, metaphysically generated, quantum paradoxes of quantum physics, which ironically are typically accepted as facts about reality. I will also show why it is very surprising that mereological nihilism is not a widely held theory, and not the premier theory in philosophy.     

A Protective Factor Against Mental Health Problems in Youths? A Critical Note on the Assessment of Self-Compassion

Self-compassion, which refers to the tendency of being kind and understanding to oneself when confronted with personal failings and difficulties, is increasingly investigated as a protective factor within the context of mental health problems. In this invited paper, I will briefly introduce the concept of self-compassion and give an overview of the research that has examined its relationship with psychopathology in youth. Then I will make my critical point regarding the assessment of self-compassion: the scales that are currently used for measuring this construct include a large number of reversely scored, negative items that measure the precise opposite of having compassion with oneself. I present evidence (partly on the basis of own data) that these negative items do not reflect the true protective nature of self-compassion and tend to inflate the relation with psychopathology. My recommendation is to remove the negative items from the scales and to assess self-compassion by means of a set of items that truly reflect its protective nature.     

A series of lower bounds to the reliability of a test

Two well-known lower bounds to the reliability in classical test theory, Guttman's ₂ and Cronbach's coefficient alpha, are shown to be terms of an infinite series of lower bounds. All terms of this series are equal to the reliability if and only if the test is composed of items which are essentially tau-equivalent. Some practical examples, comparing the first 7 terms of the series, are offered. It appears that the second term (₂) is generally worth-while computing as an improvement of the first term (alpha) whereas going beyond the second term is not worth the computational effort. Possibly an exception should be made for very short tests having widely spread absolute values of covariances between items. The relationship of the series and previous work on lower bound estimates for the reliability is briefly discussed.The authors are obliged to Henk Camstra for providing a computer program that was used in this study.     

Effects of the testing situation on item responding: cause for concern

The effects of faking on personality test scores have been studied previously by comparing (a) experimental groups instructed to fake or answer honestly, (b) subgroups created from a single sample of applicants or nonapplicants by using impression management scores, and (c) job applicants and nonapplicants. In this investigation, the latter 2 methods were used to study the effects of faking on the functioning of the items and scales of the Sixteen Personality Factor Questionnaire. A variety of item response theory methods were used to detect differential item/test functioning, interpreted as evidence of faking. The presence of differential item/test functioning across testing situations suggests that faking adversely affects the construct validity of personality scales and that it is problematic to study faking by comparing groups defined by impression management scores.     

A Pluralist Reconstruction of Confucian Democracy

In this paper, I attempt to revamp Confucian democracy, which is originally presented as the communitarian corrective and cultural alternative to Western liberal democracy, into a robust democratic political theory and practice that is plausible in the societal context of pluralism. In order to do so, I first investigate the core tenets of value pluralism with reference to William Galston??s political theory, which gives full attention to the intrinsic value of diversity and human plurality particularly in the modern democratic context. I then construct a political theory of Confucian pluralist democracy by critically engaging with two dominant versions of Confucian democracy??Confucian communitarian democracy and Confucian meritocratic democracy. My key argument is threefold: (1) the unity in Confucian democracy should be interpreted not as moral unity but as constitutional unity; (2) Confucian virtues should be differentiated (or pluralized) between moral virtues and civic virtues; (3) in Confucian democracy minorities have the constitutional right to contest public norms in civil society.     

Classical and Modern Methods of Psychological Scale Construction

Scale construction is a growth enterprise in the psychological literature. Unfortunately, many measures promise much but are severely limited by the inadequacies of their conceptualization and execution. In this paper, a model for developing psychological scales is presented that is rooted in the traditions of construct validity and classical test theory but informed by modern psychometric methods. Construct validity is conceptualized as a guiding principle in each of three phases of scale development, focused on (i) construct conceptualization and development of the initial item pool, (ii) item selection and structural validity, and (iii) assessment of external validity vis‐à‐vis other measures and relevant nontest criteria.