Seeking a Balance Between the Statistical and Scientific Elements in Psychometrics

In this paper, I will review some aspects of psychometric projects that I have been involved in, emphasizing the nature of the work of the psychometricians involved, especially the balance between the statistical and scientific elements of that work. The intent is to seek to understand where psychometrics, as a discipline, has been and where it might be headed, in part at least, by considering one particular journey (my own). In contemplating this, I also look to psychometrics journals to see how psychometricians represent themselves to themselves, and in a complementary way, look to substantive journals to see how psychometrics is represented there (or perhaps, not represented, as the case may be). I present a series of questions in order to consider the issue of what are the appropriate foci of the psychometric discipline. As an example, I present one recent project at the end, where the roles of the psychometricians and the substantive researchers have had to become intertwined in order to make satisfactory progress. In the conclusion I discuss the consequences of such a view for the future of psychometrics.     

An Introduction to Network Psychometrics: Relating Ising Network Models to Item Response Theory Models

In recent years, network models have been proposed as an alternative representation of psychometric constructs such as depression. In such models, the covariance between observables (e.g., symptoms like depressed mood, feelings of worthlessness, and guilt) is explained in terms of a pattern of causal interactions between these observables, which contrasts with classical interpretations in which the observables are conceptualized as the effects of a reflective latent variable. However, few investigations have been directed at the question how these different models relate to each other. To shed light on this issue, the current paper explores the relation between one of the most important network models—the Ising model from physics—and one of the most important latent variable models—the Item Response Theory (IRT) model from psychometrics. The Ising model describes the interaction between states of particles that are connected in a network, whereas the IRT model describes the probability distribution associated with item responses in a psychometric test as a function of a latent variable. Despite the divergent backgrounds of the models, we show a broad equivalence between them and also illustrate several opportunities that arise from this connection.     

基于认知设计系统的几何类比推理测验的编制及认知模型分析

认知诊断是一种新的测量范式。本研究首先根据Mulholland,Pellegrino＆Glaser和Whitely＆Schneider关于类比推理的研究结果,采用认知设计系统编制了几何类比推理测试题,并对所编制的测试题的质量进行验证,然后采用口语报告法分析解决几何类比推理所涉及的认知模型,结果表明：1）编制的几何类比推理测试题的质量比较好;2）学生解决类比推理所涉及的认知属性主要为置换、翻转、旋转、数量、大小、颜色与条纹和形状。     

Assessing Parenting Skills and Attitudes: A Review of the Psychometrics of Parenting Measures

There are many measures that assess parenting skills or practices, such as behavior, beliefs, coping mechanisms, reactions to stress, or discipline. However, little is known about the psychometric quality of these parenting measures. This information is essential for practitioners and researchers to aide in the selection of the most valid and reliable measures to assess parenting behavior or attitudes. This study examined the psychometric quality among parenting measures published from 1985 to 2009. After the initial search 164 measures were identified, but were reduced to 25 measures that supplied some degree of psychometric information, were published in the United States or Canada, and were in English. Measures were compared across numerous categories including respondent type, norming data, administration type, and ten psychometric variables such as internal consistency, content validity, and predictive validity. Out of the 25 measures, seven had no acceptable psychometric properties, seven had only 1–2 acceptable ratings on psychometric properties, six had between 3 and 4 acceptable psychometric ratings, none had between 5 and 6 acceptable ratings, and only five had strong psychometric properties in seven or more of the 10 categories. Likewise, only five measures provided and norming information and 14 measures provided scoring procedures. Implications, limitations, and recommendations for future research are discussed.     

心理测量本身及其在应用中的问题

从三个维度对心理测量进行了探讨：由理论基础上的缺陷所导致的局限性、在应用中所带来的问题以及由于运用统计相关分析所带来的弊端。最后对克服这些局限性的其它方法做简单介绍。     

Psychometrics: From practice to theory and back

The paper surveys 15 years of progress in three psychometric research areas: latent dimensionality structure, test fairness, and skills diagnosis of educational tests. It is proposed that one effective model for selecting and carrying out research is to chose one's research questions from practical challenges facing educational testing, then bring to bear sophisticated probability modeling and statistical analyses to solve these questions, and finally to make effectiveness of the research answers in meeting the educational testing challenges be the ultimate criterion for judging the value of the research. The problem-solving power and the joy of working with a dedicated, focused, and collegial group of colleagues is emphasized. Finally, it is suggested that the summative assessment testing paradigm that has driven test measurement research for over half a century is giving way to a new paradigm that in addition embraces skills level formative assessment, opening up a plethora of challenging, exciting, and societally important research problems for psychometricians.     

The Ethics of Immigration Revisited: Response to Brock,Fabre, Risse and Song

To a large extent, the differences between my four interlocutors and me have more to do with the way we choose to frame a question or approach a problem than with substantive disagreements. In her discussion of temporary workers and the brain drain, Gillian Brock implicitly assumes a different background framework of moral responsibility from the one I adopt in my book. Similarly, Cécile Fabre asks important questions about the intersection of immigration and criminal justice, but ones that I chose not to pursue in quite the same way or, in some cases, at all. Matthias Risse says that political theory should be ‘action‐guiding’, and I try to problematize that claim, at least to the extent that it limits the questions we can ask. Finally, I applaud the attention that Sarah Song brings to the link between political community and social membership but resist her suggestion that this shows that political community is more fundamental than social membership. I also suggest the need to clarify further the limits to democratic self‐determination.     

Indelible

Many years ago I grew away from the evangelical Christian faith that had grounded my life (before and beyond death) since my early teens. Or so I thought: the stories my body now tell confront me with the sense that I have – secretly, ambivalently – held on to elements of that faith. Over recent times, through and since my doctoral studies, I have embraced poststructural and Deleuzian sensibilities. These, one might think, run right up against the entrenched binaries and certainties that remain indelibly inscribed. The narrative of progress and development I have been telling myself over the decades – that I have not just grown away but grown up – is no longer tenable. In this paper I examine my doubt at whether I doubt. Amongst the most disturbing stories is one of being beaten in God's name. Its scars remain. I revisit this story in an attempt to dwell more fully in the pain (and pleasure?) of cane on flesh. How am I to (at)tend (to) those scars? What are their meanings? I draw from the psychodynamic and poststructural theoretical frameworks that seem to have failed me, in inquiring into the political, cultural, emotional, psychological and spiritual processes at play in this current disturbance.     

Measuring sport fantasy proneness and deflated reality in sport and performance: Development and validation of two context-specific instruments

Fantasy-prone personalities and generalized states of pessimism and hopelessness have been linked to various behavioural, cognitive, and health related outcomes in the general population. Nevertheless, to date, knowledge of sport-specific fantasy proneness and athletes’ perception of deflated reality in sport is scarce, possibly due to a lack of appropriate psychometric instruments to examine these two important context-specific constructs. In this research, we developed the Sport Fantasy Proneness Scale (SFPS) and the Deflated Reality in Sport Scale (DRSS), first assessing the content validity of items for the instruments following a rigorous process. Through two cross-sectional samples (Study 1: Sample 1 N = 255, Sample 2 N = 260) and one longitudinal sample (Study 2: Sample 3 N = 118) of competitive athletes in the UK, we then examined factorial, convergent, discriminant, and concurrent validity and measurement invariance (Study 1), as well as internal consistency, and test-retest reliability (Study 2) of the two new scales. Results revealed robust construct validity and reliability of scores on the two instruments and suggested very good invariance when implementing the new scales to athletes from different sports and competitive levels and good invariance for cross-gender comparisons. The new scales fill a gap in the sport psychology literature and provide researchers and practitioners with robust psychometric instruments to examine new research questions and tackle issues relevant to athletes’ sport-specific fantasy proneness and perceptions of deflated reality.     

Adaptation of Nussbaum's Capabilities Framework to Community Mental Health: A Consumer‐Based Capabilities Measure

The capabilities approach provides a rich evaluative framework to guide transformative change in the community mental health system. This study reports the content and construct validity and psychometric properties of a contextualized measure of the extent to which mental health programs foster achieved capabilities. The Achieved Capabilities Questionnaire for Community Mental Health (ACQ‐CMH), adapted from Nussbaum's capabilities framework, was developed previously with consumer collaboration. Content validity was assessed through a collaborative process, involving a panel of eight consumers, staff members, and senior researchers. The resulting shorter version (ACQ‐CMH‐98) was completed by 332 community mental health consumers sampled throughout Portugal. Factor (PCA) analysis, internal consistency reliability, and test–retest reliability over 2 weeks (N = 33) showed good psychometric properties. The resulting six‐factor structure with 48 items explains 48.88% of the total variance (KMO = 0.89; Bartlett p = .00). Internal consistency of the obtained dimensions ranges from .91 to .76. Associations of the measure with recovery, quality of life, and psychological distress scales add further evidence of construct validity. The adaptation of Nussbaum's framework stressed specific components that may enhance understanding and change within the community mental health system.     

Forethoughts from the editor

Conclusion  Perhaps the fundamental question involved in assessing research can best be summarized as follows: On the basis of what criteria must the long-term accountability of public funds allocated for research be measured, and who should determine these criteria? Should these criteria be based on the quality or the quantity of research produced; on the actual utilization of the research results; on the relevance of research to one or more aspects of “development”; or on the medium- or long-range “potential” impact of research results? Should these criteria be in accordance with “international” standards or reflect primarily national goals and objectives? In addition, there is the complex issue of knowledge integration, an understanding of which is a prerequisite for the generation of a “generic frame-work” designed to focus on the organization of communication between science and the social environment. Given that research and knowledge generation are becoming increasingly international in scope and impact, it is insufficient to merely contend that knowledge integration must occur through various forms of collaboration involving cross fertilization and interdisciplinary perspectives. This is already complicated at the national—let alone the international—level. Unfortunately, none of these issues have thus far been solved by the ever-expanding range of tools available for assessing research.     

Psychometrics properties of the Swansea Muscularity Attitudes Questionnaire (SMAQ)

The psychometric soundness of the Swansea Muscularity Attitudes Questionnaire (SMAQ) was examined using two independent samples of Canadian male post-secondary students (Ns = 250 and 310, respectively). Exploratory and confirmatory factor analyses suggested that the final 19-item version of the SMAQ is represented by three latent constructs, which we have labeled: intention to become more muscular (eight-items), positive attributes of muscularity (nine-items), and engagement in muscle-building activities (two-items). Alpha coefficients for the intention and positive subscales were very good (i.e., range = .86–.92). Correlation coefficients between the two-items comprising the engagement subscale also suggest that they can be combined into a composite measure (rs = .70 [study 1] and .58 [study 2]). Various tests of criterion-related and construct validity were conducted, with results indicating that the SMAQ subscales are valid indicants of the drive for muscularity.     

关于实验心理学中推理的现代心理测量方法(英文)

推理测验与推理实验不同。推理测验用来衡量智商;而推理实验则用来研究推理的心理过程。测验旨在找出能力的个体差异;而实验则意在突出所有个体的一般性心理过程。近年来,心理学中的推理已经发生了变化,我们在理论进化方面简要地总结了如下这些变化:关注点从推理任务中的反应过程变为个体对不同任务做反应时的差异和其中稳定的反应模式,开始考虑不同推理者的组织差异和数量差异(犹如不同游泳运动员的游泳动作和肌肉数量)。然而,这些理论创新必须有精确的建模技术的支持。我们将以条件推理材料为例,展示了现代心理测量方法可以帮我们解决理论创新中产生的三类主要的研究问题。     

Some old and new directions in Rorschach testing

The perennial dispute between clinicians and researchers as to the validity of the Rorschach is discussed. It is suggested both groups are correct in that the Rorschach provides both poor psychometric data as seen by the researcher and rich idiographic data as seen by the clinician. The freedom of the Rorschach response process is viewed as largely responsible for both these situations. Some possible new directions for Rorschach usage and research are discussed which may improve on the quality of the idiographic information provided by the Rorschach test.     

The Identity Status Approach: In Need of Fundamental Revision and Qualitative Change

This article is a response to the commentaries of Waterman (this issue) and Berzonsky and Adams (this issue) on my review of the identity status field (van Hoof, this issue). The commentaries illustrate that the identity status researchers attribute different meanings to the identity statuses and identity status research; while Waterman finds it difficult to recognize the field in my review, Berzonsky and Adams agree with several of my conclusions. The two commentaries represent two pillars on which the status approach rests. The difference between the review and the commentaries is that the first includes both pillars, whereas each commentary consists primarily of a selective presentation of assumptions and data. This response focuses on two main points of the commentaries: (a) are the statuses sufficiently validated? and (b) to what extent are they suitable for testing developmental identity hypotheses? Using specific examples I show that the line of reasoning in the commentaries and the data proposed in support of these arguments in fact often support the conclusions of my review, for I demonstrate that the statuses are insufficiently validated and that the statuses are unsuitable for measuring identity development. These conclusions call for a fundamental revision of the identity status approach and a qualitative change in identity theory and research. Finally, the alternative approaches presented by Berzonsky and Adams are evaluated with respect to their contribution to an indispensable qualitative change in the identity status field.     

A psychometric assessment of a Swedish translation of Spreitzer's empowerment scale

The psychometric properties of a Swedish translation of Spreitzer's (1995a) empowerment scale were assessed. Three parallel assessments were made of data from 1,107 female registered nurses, 758 female assistant nurses, and 107 male registered and assistant nurses. The analyses indicated that the psychometric properties of the scale can be considered satisfactory. Suggestions are made regarding future research in the fields of psychometrics and work- and health-psychology.     

Theatre and Research in the Reproductive Sciences

This paper explores the power of theatre to engage the public and my personal journey using theatre as a research tool in reproductive science. I argue that the capacity of theatre to simultaneously engage the minds and hearts of audience members qua research participants affords audience members the capacity to provide researchers with insightful comments informed by the scientific, social and tacit knowledge derived from the performance, integrated with their lived experience. Theatre is a particularly important research strategy when investigating public understandings and desires about complex issues such as those related to reproductive and genetic science.     

Perspectives on Psychometrics Interviews with 20 Past Psychometric Society Presidents

Psychometrika - In this article, we present the findings of an oral history project on the past, present, and future of psychometrics, as obtained through structured interviews with twenty past...