The Importance of Construct Validity in Consumer Research

Vosgerau, Scopelliti, and Huh (2020) question the construct validity of the most commonly used measures of self‐control and argue that many studies that claim to be examining self‐control may in fact be examining something else. In this comment, we generalize this argument and propose that this criticism could be leveraged against many other areas of consumer research. Indeed, many studies fail to properly establish the construct validity of their measures. Consequently, they may not be examining the construct they purport to be studying, which can lead to unreliable and conflicting findings.     

The Construct Validity of Assessment Centre Performance Dimensions

Evidence regarding the construct validity of assessment centre performance dimensions is reviewed. The evidence strongly suggests that variance in ratings tends to reflect exercises more than individual performance dimensions, thus calling into question the construct validity and utility of these dimensions. A number of biases in the assessment centre process, as well as more general rating biases are noted that may be responsible for these pervasive exercise effects. Suggestions are made for enhancing the construct validity of performance dimensions.     

评价中心的结构效度研究

评价中心虽然具备很高的预测效度,但其结构效度指标却不太理想,如研究普遍发现其汇聚效度和区分效度较低。影响评价中心结构效度的因素众多,如评分维度因素（数量和类型）、评价者因素（培训方式和人员类型）、测评方法因素（情景导向特征、特质激活潜力、测评活动形式）以及系统的观察与评价程序。该文从上述因素出发,综述了评价中心结构效度的相关研究,总结了提高评价中心结构效度的措施,并指出了未来的研究方向     

The Construct Validity of the Utrecht Work Engagement Scale: Multisample and Longitudinal Evidence

This study investigated the factor structure and factorial group and time invariance of the 17-item and 9-item versions of the Utrecht Work Engagement Scale (UWES; Schaufeli et al. (2002b) Journal of Happiness Studies 3:71–92). Furthermore, the study explored the rank-order stability of work engagement. The data were drawn from five different studies (N = 9,404), including a three-year longitudinal study (n = 2,555), utilizing five divergent occupational samples. Confirmatory factor analysis supported the hypothesized correlated three-factor structure—vigor, dedication, absorption—of both UWES scales. However, while the structure of the UWES-17 did not remain the same across the samples and time, the structure of the UWES-9 remained relatively unchanged. Thus, the UWES-9 has good construct validity and use of the 9-item version can be recommended in future research. Moreover, as hypothesized, Structural Equation Modeling showed high rank-order stabilities for the work engagement factors (between 0.82 and 0.86). Accordingly, work engagement seems to be a highly stable indicator of occupational well-being.     

面试预测效度和构想效度研究述评

面试是人才甄选中最常用的测量工具。大量研究证实,面试的预测效度比较理想,但不同类型面试的预测效度存在差异。虽然能够证实面试的预测效度较好,但对面试的测量构想却知之甚少。研究面试的构想效度,对于提高面试的递增效度有着重要的实践价值。相对人格成分而言,以往研究对面试能够测量到认知成分形成了更加一致地认识。     

效度概念演进及其新发展

回顾了心理测量学中效度概念发展的三个阶段,早期效度概念关注的是两个变量之间的相关,随后重点转向效度的多种类型,现阶段则注重考察测验的构想效度.总之,效度确证成为一个持续不断的动态过程;在这个过程中,运用各种技术不断地评价、质疑和检查由测验分数所作推论和解释的有效性成为心理与教育测量研究中不可或缺的组成部分.     

Effects of a Fluency-Building Program on the Reading Performance of Low-Achieving Second and Third Grade Students

This study evaluated the effects of a fluency-based reading program with 15 second and third grade students and 15 matched controls. Gains in oral reading fluency on untrained CBM probes were evaluated using a matched-pairs group-comparison design, whereas immediate and two-day retention gains in oral reading fluency on trained passages were evaluated using an adapted changing criterion design. Increases in WRCM due to training and number of trainings to criterion were also evaluated as a function of pre-training fluency levels. Results showed statistically significant gains on dependent measures for the treatment group, mean increases of two to three grade levels in passages mastered, and an optimal pre-training fluency range of 41-60 WRCM. Implications for fluency-based reading programs are discussed.     

The Clinician Affective Reactivity Index: Validity and Reliability of a Clinician-Rated Assessment of Irritability

Irritability is impairing in youth and is the core feature of disruptive mood dysregulation disorder (DMDD). Currently, there are no established clinician-rated instruments to assess irritability in pediatric research and clinical settings. Clinician-rated measures ensure consistency of assessment across patients and are important specifically for treatment research. Here, we present data on the psychometric properties of the Clinician Affective Reactivity Index (CL-ARI), the first semistructured interview focused on pediatric irritability. The CL-ARI was administered to a transdiagnostic sample of 98 youth (M age = 12.66, SD = 2.47; 41% female). With respect to convergent validity, CL-ARI scores were (a) significantly higher for youth with DMDD than for any other diagnostic group, and (b) showed uniquely strong associations with other clinician-, parent-, and youth-report measures of irritability compared to measures of related constructs, such as anxiety. The three subscales of the CL-ARI (temper outbursts, irritable mood, impairment) showed excellent internal consistency. Test-retest reliability of the CL-ARI was adequate. These data support that irritability can be feasibly, validly, and reliably assessed by clinicians using the CL-ARI. A validated, gold-standard assessment of pediatric irritability is critical in advancing research and treatment efforts.     

The Construct Validity of the Work-Related Flow Inventory in a Sample of Australian Workers

The purpose of this study was to examine the construct validity of the WOrk-reLated Flow inventory (WOLF; Bakker, 2008). This instrument was administered to 711 men and women who were working in Queensland, Australia. The results from the confirmatory factor analysis showed that the WOLF has moderately acceptable construct validity, with the three-factor model being a borderline fit to the data. Tests of the convergent validity of the WOLF yielded satisfactory results. However, the analysis of the discriminant validity of the WOLF showed that the instrument poorly discriminated between work enjoyment and intrinsic work motivation. Follow-up exploratory factor analysis, using recommended procedures for determining the number of factors to extract, revealed a two-factor solution, with the work enjoyment and intrinsic work motivation items loading on the same factor. Drawing on literature on psychological flow and motivation, as well as the findings of the present study, questions are raised over the adequacy of the conceptual basis of the three-factor model of work-related flow, the discriminant validity of the WOLF subscales, and the appropriateness of the wording of several of this measure's items. Using alternative methods and measures to investigate flow in work settings is recommended.     

Construct validity of the Sivik Psychosomaticism test and test of Operational style: Correlations with four Minnesota Multiphasic Personality Inventory (MMPI) subscales

To evaluate the construct validity (convergent and divergent) of the Sivik Psycho Somaticism test (SPS) and test of Operationality (OPER), Pearson correlation coefficients between SPS scales and subscales, OPER and Minnesota Multiphasic Personality Inventory (MMPI) subscales Hypochondria (Hs), Depression (D), Hysteria (Hy) and Alexithymia (Al) were calculated. Eighty-eight healthy individuals and 285 psychosomatic patients completed the SPS and OPER tests and MMPI; Hs, D, Hy and Al. The results show that most of the SPS subscales and OPER are significantly correlated to several MMPI subscales in both a normal and a psychosomatic population. The results are in concordance with the theoretical hypotheses and confirm the validity of the SPS and OPER constructs.     

An assessment of the similarities between a measure of positive perfectionism and a measure of conscientious achievement striving

Objectives

The purpose of the study was to examine the similarities between the constructs measured by the positive perfectionism subscale from the Positive and Negative Perfectionism Scale and the achievement striving subscale from the Revised NEO Personality Inventory.

Design

A non-experimental design was adopted.

Method

One-hundred and seventy-nine junior basketball players (age M = 16.50, s = 1.12) completed measures of positive perfectionism, conscientious achievement striving and other external measures.

Results

Analyses revealed that the two scales were highly positively correlated and demonstrated a similar pattern of relationships with the external measures. However, a single latent factor model provided a comparatively poorer fit than a two latent factor model.

Conclusions

There is some evidence that the positive perfectionism and achievement striving constructs measured by the scales in this study are distinguishable as two distinct factors in a confirmatory factor analysis; however, more empirical evidence is needed to establish their substantive differences.     

The Scale for Assessing Emotional Disturbance: Concurrent Validity

The Scale for Assessing Emotional Disturbance (SAED) is a standardized, norm-referenced measure designed to operationalize the federal definition of Emotional Disturbance. It has demonstrated content validity and reliability. The two studies reported here address the SAED's concurrent validity. Samples in each study consisted of students identified with Emotional Disturbance. In both studies, special education teachers rated students on the SAED and a second measure of children's emotional and behavior problems. In the first study the SAED was compared to the Revised Behavior Problem Checklist. Correlations were generally moderate to high for similar subscales and low for dissimilar subscales. In the second study, the SAED was compared to the Teacher Report Form. Again, correlations generally were as expected and compatible with results of the first study. Results indicate that the SAED emotional and behavior problem subscales measure constructs akin to emotional and behavior problems measured by similar collections of item on the Revised Behavior Problem Checklist and Teacher Report Form. Implications of results and recommendations for use of the SAED are discussed.     

正念注意觉知量表儿童版~MAAS-C在中国城市及农村儿童中使用的信效度检验

目的:修订中文版正念注意觉知量表儿童版(the Mindful Attention Awareness Scale—Children,MAAS-C),考察其在中国文化背景下的适用性及信效度。方法:城市及农村地区共619名小学4-6年级学生参与研究。对数据进行项目分析、探索性因素分析、验证性因素分析、信度分析、效标关联效度分析、性别及城乡地区等值性检验。结果:中文版MAAS-C为单维结构,具有良好的结构效度、效标效度及信度。测量不变性检验表明,该量表在不同性别儿童间达到严格等值,城乡地区儿童间达到弱等值。结论:正念觉知量表儿童版的中文版本各项心理测量学指标良好,适宜在我国儿童群体中使用。     

Assessing the validity of multinomial models using extraneous variables: An application to prospective memory

The class of multinomial processing tree (MPT) models has been used extensively in cognitive psychology to model latent cognitive processes. Critical for the usefulness of a MPT model is its psychological validity. Generally, the validity of a MPT model is demonstrated by showing that its parameters are selectively and predictably affected by theoretically meaningful experimental manipulations. Another approach is to test the convergent validity of the model parameters and other extraneous measures intended to measure the same cognitive processes. Here, we advance the concept of construct validity (Cronbach & Meehl, 1955 Cronbach, L. J. and Meehl, P. E. 1955. Construct validity in psychological tests. Psychological Bulletin, 52: 281–302. [Crossref], [PubMed], [Web of Science ®] , [Google Scholar]) as a criterion for model validity in MPT modelling and show how this approach can be fruitfully utilized using the example of a MPT model of event-based prospective memory. For that purpose, we investigated the convergent validity of the model parameters and established extraneous measures of prospective memory processes over a range of experimental settings, and we found a lack of convergent validity between the two indices. On a conceptual level, these results illustrate the importance of testing convergent validity. Additionally, they have implications for prospective memory research, because they demonstrate that the MPT model of event-based prospective memory is not able to differentiate between different processes contributing to prospective memory performance.     

中学生问题行为多视角评估的构想效度

本文以多质多法和验证性因素分析的方法对由中学生问题行为三视角评估(自我评定、教师评定和家长评定)构成的数据模型进行了构想效度的研究。研究样本为分层随机抽取的北京市292名中学生,对其三视角问题行为的施测结果进行了会聚效度和区分效度的分析,结果表明:(1)中学生问题行为三视角六因子的MTMM模型构想效度良好;(2)评定者对问题行为的评估有影响,存在评定者效应,而且评定者效应对内倾问题因子比对外倾问题因子影响大。     

Factors which Improve the Construct Validity of Assessment Centers: A Review

This article reviews 21 studies which manipulated specific variables to determine their impact on the construct validity of assessment centers. This review shows that the studies regarding the impact of different observation, evaluation, and integration procedures yielded mixed results. Conversely, dimension factors (number, conceptual distinctiveness, and transparency), assessor factors (type of assessor and type of assessor training), and exercise factors (exercise form and use of role-players) were found to moderate construct validity. On the basis of the review, practical recommendations are derived to maximize the probability that practitioners design and administer an assessment center with construct validity. Finally, new perspectives for future research are identified.     

中学生自我导向学习倾向性量表的结构和效度

本研究的目的是修订中学生自我导向学习倾向性量表,探讨它的结构和效度。研究参考台湾修订版对Guglielmino量表进行翻译,构成量表;结合内外控量表、学业成就归因量表和学业自我效能感量表,有效测试了1000名中学生,采用信度分析、因素分析和外部效标分析等方法考察量表的结构和效度。结果发现6个因子结构最优,α分别为0.861,0.791,0.809,0.630,0.888,0.781,各个条目的因子负荷介于0.30-0.69。6因子结构模型的各项拟合指标都在0.90以上,6因子与各个效标的相关为大多达到了显著性水平。研究的结论是修订的中学生自我导向学习倾向量表具有良好的6因子结构和信效度。     

Incremental Validity of Cognitions in a Clinical Case Formulation: An Intraindividual Test in a Case Example

Incremental validity, the ability of a measure to predict or explain variance over and above other measures, is an important psychometric characteristic of standardized measures, but has received little attention idiographically. Idiographic assessment may be an important part of developing a clinical case formulation, guiding treatment by developing an individualized understanding of the variables that trigger and maintain distress. This study examined whether the idiosyncratic cognitive schema hypothesized by a clinician in a cognitive case formulation explained distress incrementally over that of situational triggers. Using daily ratings of situational triggers, idiosyncratic cognitions, and distress, the incremental validity of cognitions in predicting each of six distress measures was tested in a case example using dynamic time series regression. The incremental variance explained by cognitions varied across the distress measures, suggesting that, in this case example, targeting thoughts and beliefs for treatment may be important for only certain types of distress.

The Validity of the Role Construct Repertory Test as a Measure of Sexual Identification

Although the majority of research on Therapeutic Assessment (TA) discusses the application of TA in research or private practice settings, we found that the model could be applied in a community mental health setting. We argue that when implementing this model with racially diverse, low socioeconomic status children, it is essential to integrate issues of class, privilege, and race into the assessment process. A case is presented that illuminates the specific concerns and struggles of adapting this model to a community psychology population. This analysis includes the interface with systems, placement stability, and consideration of culturally responsible treatment. We also demonstrate how the support of a treatment team helps the individual clinician process and integrate the levels of trauma and pain with which these families present.