Bayesian SEM for Specification Search Problems in Testing Factorial Invariance

Specification search problems refer to two important but under-addressed issues in testing for factorial invariance: how to select proper reference indicators and how to locate specific non-invariant parameters. In this study, we propose a two-step procedure to solve these issues. Step 1 is to identify a proper reference indicator using the Bayesian structural equation modeling approach. An item is selected if it is associated with the highest likelihood to be invariant across groups. Step 2 is to locate specific non-invariant parameters, given that a proper reference indicator has already been selected in Step 1. A series of simulation analyses show that the proposed method performs well under a variety of data conditions, and optimal performance is observed under conditions of large magnitude of non-invariance, low proportion of non-invariance, and large sample sizes. We also provide an empirical example to demonstrate the specific procedures to implement the proposed method in applied research. The importance and influences are discussed regarding the choices of informative priors with zero mean and small variances. Extensions and limitations are also pointed out.     

Group Differences in Regression Intercepts: Implications for Factorial Invariance

Studies of differential prediction typically examine group differences in linear regression slopes or intercepts for predicting criterion scores from one or more test scores. When there are no group differences in slopes, what are the implications of differences in regression intercepts for the measurement equivalence of the tests or criterion across groups? Measurement equivalence is here defined as factorial invariance under a single-factor model for the tests and criterion. Two theorems are given that describe conditions under which intercept differences can exist under factorial invariance. In such cases, intercept differences do not result from measurement bias in either the tests or criterion. The conditions of the theorems are testable using multiple-group confirmatory factor analysis. These test procedures are illustrated in real data. The implications of the theorems and the test procedures for studies of differential prediction are discussed.     

Measurement Invariance,Predictive Invariance,and the Duality Paradox

The statistical literature on bias in psychological testing distinguishes at least two forms of bias: measurement bias and predictive bias. Measurement bias concerns group differences in the relationship between a test and the latent variable to be measured. Predictive bias concerns group differences in the relationship between a test and an external criterion. How are these two forms of bias related? For example. if a test is unbiased in the predictive sense, does this fact support the hypothesis that the test is unbiased in the measurement sense? A theorem is given that describes the conditions under which measurement invariance (lack of bias) is consistent with predictive invariance for the linear case. Paradoxically, these two forms of invariance are shown to be inconsistent under realistic conditions. This duality or inconsistency is illustrated in simulated data. The implications of the duality for group differences research are illustrated in real data involving gender and ethnic differences on the SAT. The phenomenon of duality may force a reinterpretation of common empirical findings of test criterion regression slope invariance. and of invariance in test validities. Other implications are discussed.     

Factorial Structure and Measurement Invariance of the Innovative Behavior Questionnaire

The article presents an analysis of the factorial structure and measurement invariance of the Innovative Behavior Questionnaire, developed by Scott and Bruce. Although the instrument is widely used to capture individuals' innovative behavior, very little evidence concerning its psychometric properties is available. A time‐lagged study among 382 employees was conducted to check the factorial structure of the questionnaire, using confirmatory factor analysis, as well as its measurement invariance across gender and time. One‐factor structure (with correlated error terms of first three items) and strict invariance across time and across gender of the Innovative Behavior Questionnaire were demonstrated. As such, the measure can be used as a reliable tool for capturing individuals' innovative behavior by self‐report.     

What Invariance Is and How to Test for It

Causal assessment is the problem of establishing whether a relation between (variable) X and (variable) Y is causal. This problem, to be sure, is widespread across the sciences. According to accredited positions in the philosophy of causality and in social science methodology, invariance under intervention provides the most reliable test to decide whether X causes Y. This account of invariance (under intervention) has been criticised, among other reasons, because it makes manipulations on the putative causal factor fundamental for the causal methodology; consequently, the argument goes, the account is ill-suited to those contexts where manipulations are not performed, for instance, the social sciences. The article aims to extend the account of invariance (under intervention), in a way that manipulations on the putative causal factors are not methodologically fundamental, and yet invariance remains key for causal assessment both in experimental and non-experimental contexts.     

Measurement and Predictive Invariance of a Work–Life Boundary Measure Across Gender

Pervasive gender norms influence the way in which men and women create and experience work–life boundaries and work–life conflict, which may lead to differing conceptions of work–life boundary management. Measurement equivalence/invariance (ME/I) and predictive invariance analyses were conducted on the WorkLife Indicator^TM (WLI), a measure of work–life boundary management, to determine if men and women maintain similar conceptualizations of work–life boundary constructs and if work–life outcomes are predicted equally well by WLI scales across gender. Three of the five WLI scales exhibited ME/I; however, half of the WLI scale–criterion pairings failed to demonstrate predictive invariance. These results indicate that measurement and/or predictive invariance cannot be assumed when using work–life boundary assessments across gender.     

Using a Few Snapshots to Distinguish Mountains from Waves: Weak Factorial Invariance in the Context of Trait-State Research

In this article, we show that the underlying dimensions obtained when factor analyzing cross-sectional data actually form a mix of within-person state dimensions and between-person trait dimensions. We propose a factor analytical model that distinguishes between four independent sources of variance: common trait, unique trait, common state, and unique state. We show that by testing whether there is weak factorial invariance across the trait and state factor structures, we can tackle the fundamental question first raised by Cattell; that is, are within-person state dimensions qualitatively the same as between-person trait dimensions? Furthermore, we discuss how this model is related to other trait-state factor models, and we illustrate its use with two empirical data sets. We end by discussing the implications for cross-sectional factor analysis and suggest potential future developments.     

Longitudinal Predictive Validity of a Learning Potential Test

The aim of this article is to provide empirical psychometric evidence of the (longitudinal) predictive validity of a learning potential measure—the Learning Potential Computerised Adaptive Test (LPCAT)—in comparison with standard static tests with school aggregate results as the criterion measure. Participants were 79 boys (mean age 12.44, SD = 0.44) and 72 girls (mean age 11.18, SD = 0.42) attending two private schools. Correlation and regression analyses were used to evaluate the predictive validity of the learning potential and standard test scores for school aggregate academic results as criterion measure. Results indicate that learning potential scores were statistically significant predictors of aggregate academic results and provided results that were comparable to those of the standard test results—providing empirical support for the use of learning potential tests in mainstream educational settings.     

The Maslach Burnout Inventory: Validating Factorial Structure and Invariance Across Intermediate,Secondary, and University Educators

The purposes of the study were: (a) to test for the factorial validity of the Maslach Burnout Inventory (MBI), for 543 teachers at the intermediate (n = 163), secondary (n = 162), and university (n = 218) levels, and (b) to test for the equivalence of factorial measurements and structure across groups. Initial confirmatory factor analysis of the hypothesized 3-factor structure yielded a malfitting model for each group of educators. With a view to improving the MBI for use with educators, subsequent exploratory and confirmatory factor analyses resulted in the deletion of four scale items. Tests for invariance revealed the equivalency of remaining items across intermediate and secondary teachers, and items measuring Emotional Exhaustion and Depersonalization across all three groups; the structure of burnout was only partially invariant across educators. The study has important implications for substantive studies focusing on multigroup comparisons across teaching panels.     

Assessing Alexithymia: Psychometric Properties and Factorial Invariance of the 20-Item Toronto Alexithymia Scale in Nonclinical and Psychiatric Samples

The 20-item Toronto Alexithymia Scale (TAS-20) is a self-report questionnaire designed to measure the three components of alexithymia; difficulty identifying feelings in the self (DIF), difficulty describing feelings (DDF), and externally orientated thinking (EOT). We examined the scale’s psychometric properties in Australian nonclinical (N = 428) and psychiatric (N = 156) samples. In terms of factorial validity, confirmatory factor analyses found the traditional 3-factor correlated model (DIF, DDF, EOT) to be the best and most parsimonious solution, but it did not reach adequate levels of goodness-of-fit in either sample. Several EOT items loaded poorly on their intended factor, and a reverse-scored item method factor was present; the factor structure of the scale was invariant across both samples. A higher-order factor model (with a single higher-order factor) was slightly inferior to the correlated models, but still tenable. The total scale score and DIF and DDF subscales displayed sound internal consistency, but the EOT subscale did not. We conclude that the TAS-20 has, for the most part, adequate psychometric properties, though interpretation should focus only on the total scale score and DIF and DDF subscales; we recommend the EOT subscale score not be used. Implications for clinical use and future revision of the scale are discussed.     

平均数差异显著性检验统计检验力和效果大小的估计原理与方法

该文以平均数差异显著性检验为例,对实验数据进行假设检验后,继续对其统计检验力和效果大小进行估计的基本原理和方法作一介绍。     

Consumer Preferences for the Predictive Genetic Test for Alzheimer Disease

The purpose of this study was to assess consumer preferences for predictive genetic testing for Alzheimer disease in the United States. A rating conjoint analysis was conducted using an anonymous online survey distributed by Qualtrics® to a general population panel in April 2011 in the United States. The study design included three attributes: Accuracy (40 %, 80 %, and 100 %), Treatment Availability (Cure is available/Drug for symptom relief but no cure), and Anonymity (Anonymous/Not anonymous). A total of 12 scenarios were used to elicit people’s preference, assessed by an 11-point scale. The respondents also indicated their highest willingness-to-pay (WTP) for each scenario through open-ended questions. A total of 295 responses were collected over 4 days. The most important attribute for the aggregate model was Accuracy, contributing 64.73 % to the preference rating. Treatment Availability and Anonymity contributed 20.72 % and 14.59 %, respectively, to the preference rating. The median WTP for the highest-rating scenario (Accuracy 100 %, a cure is available, test result is anonymous) was $100 (mean?=?$276). The median WTP for the lowest-rating scenario (40 % accuracy, no cure but drugs for symptom relief, not anonymous) was zero (mean?=?$34). The results of this study highlight attributes people find important when making the hypothetical decision to obtain an AD genetic test. These results should be of interests to policy makers, genetic test developers and health care providers.     

Predictive validity of a Rashevsky-Landahl neural net: Test of a model of masking for form

Ss were asked to search for a single-letter target embedded in a context of (1) letter strings that were orthographically well structured though not English words, or (2) letter strings that were poorly structured and unpronounceable. Neither college students nor fifth graders differed in scanning rate as a function of the context structure. Reports from Ss revealed that orthographic structure was frequently noted but was used inconsistently, if at all, and without benefit to the search. It was concluded that optimal strategies for processing verbal features change with the task and that pickup of larger structural units is incompatible with fut letter search.     

不同定义平行测验等值的群体不变性

群体不变性是等值的一个重要假设,即对不同的考生子群体等值函数一致。本研究对不同平行测验定义下线性等值的群体不变性进行了理论分析和模拟研究,模拟研究REMSD指标通过六种不同加权方式计算。结果显示,严格平行测验在信度较低时REMSD指标更大;子群体均值差异和信度差异对REMSD的影响存在明显的交互作用;REMSD指标在期望权重等权下的最大,在分数权重采用子群体比例加权最小。最后对结果进行了讨论,对REMSD权重使用及进一步研究给出了建议。     

A Statistical Interpretive Method for Neuropsychological Test Data

The accurate interpretation of large numbers of neuropsychological tests within a flexible battery approach is a difficult and sometimes controversial process. We present a statistically based method of interpretation (Rohling's Interpretive Method or RIM) and evaluation of neuropsychological data that allows for varying numbers of tests along a varying number of cognitive domains, yet remains psychometrically based. This method requires informed clinical judgment in that the level of confidence for tests, cognitive domains, and global indices are used as the backdrop for interpretive decisions. Specific procedures for use are presented in a systematic, detailed fashion to allow the interested reader to replicate the method. Two case examples are presented: a straightforward case of cerebrovascular insult and a more complicated case of mixed etiology. Examples include a variety of different neuropsychological tests commonly used in a flexible battery approach. A discussion of the practicality, ease of use, and potential limitations of this method are further presented.     

Rosenberg Self-Esteem Scale: Method Effects,Factorial Structure and Scale Invariance Across Migrant Child and Urban Child Populations in China

Using confirmatory factor analyses, this study examined the method effects on a Chinese version of the Rosenberg Self-Esteem Scale (RSES; Rosenberg, 1965 Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.[Crossref] , [Google Scholar]) in a sample of migrant and urban children in China. In all, 982 children completed the RSES, and 9 models and 9 corresponding variants were specified and tested. The results indicated that the method effects are associated with both positively and negatively worded items and that Item 8 should be treated as a positively worded item. Additionally, the method effects models were invariant across migrant and urban children in China.     

Predictive validity of an Implicit Association Test for assessing anxiety

The Implicit Association Test (IAT) was adapted to measure anxiety by assessing associations of self (vs. other) with anxiety-related (vs. calmness-related) words. Study 1 showed that the IAT-Anxiety exhibited good internal consistency and adequate stability. Study 2 revealed that the IAT-Anxiety was unaffected by a faking instruction. Study 3 examined the predictive validity of implicit and explicit measures and showed that the IAT-Anxiety was related to changes in experimenter-rated anxiety and performance decrements after failure. Study 4 found that several behavioral indicators of anxiety during a stressful speech were predicted by the IAT. Taken together, these studies show that the IAT-Anxiety is a reliable measure that is able to predict criterion variables above questionnaire measures of anxiety and social desirability.