Multiple Imputation of Item Scores in Test and Questionnaire Data,and Influence on Psychometric Results

The performance of five simple multiple imputation methods for dealing with missing data were compared. In addition, random imputation and multivariate normal imputation were used as lower and upper benchmark, respectively. Test data were simulated and item scores were deleted such that they were either missing completely at random, missing at random, or not missing at random. Cronbach's alpha, Loevinger's scalability coefficient H, and the item cluster solution from Mokken scale analysis of the complete data were compared with the corresponding results based on the data including imputed scores. The multiple-imputation methods, two-way with normally distributed errors, corrected item-mean substitution with normally distributed errors, and response function, produced discrepancies in Cronbach's coefficient alpha, Loevinger's coefficient H, and the cluster solution from Mokken scale analysis, that were smaller than the discrepancies in upper benchmark multivariate normal imputation.     

Thurstonian Scaling of Compositional Questionnaire Data

To prevent response bias, personality questionnaires may use comparative response formats. These include forced choice, where respondents choose among a number of items, and quantitative comparisons, where respondents indicate the extent to which items are preferred to each other. The present article extends Thurstonian modeling of binary choice data to “proportion-of-total” (compositional) formats. Following the seminal work of Aitchison, compositional item data are transformed into log ratios, conceptualized as differences of latent item utilities. The mean and covariance structure of the log ratios is modeled using confirmatory factor analysis (CFA), where the item utilities are first-order factors, and personal attributes measured by a questionnaire are second-order factors. A simulation study with two sample sizes, N = 300 and N = 1,000, shows that the method provides very good recovery of true parameters and near-nominal rejection rates. The approach is illustrated with empirical data from N = 317 students, comparing model parameters obtained with compositional and Likert-scale versions of a Big Five measure. The results show that the proposed model successfully captures the latent structures and person scores on the measured traits.     

问卷数据建模的前传

问卷法是一种常见的实证研究方法。问卷数据建模的前期工作,就像是一栋大楼的奠基工程,基础是否扎实,影响后续的工程质量。本文专门讨论统计模型建立之前要做的事情（重点是量表评价）,内容包括：处理缺失值、评价量表的结构效度和题目删除的适当性、多维量表需要合成总分时检验同质性并计算合成信度、检验共同方法偏差和评价（变量）区分效度、题目打包、检验自变量的多重共线性,最后也涉及建模理据和无关变量控制等。     

Investigation of Multiple Imputation in Low-Quality Questionnaire Data

The performance of multiple imputation in questionnaire data has been studied in various simulation studies. However, in practice, questionnaire data are usually more complex than simulated data. For example, items may be counterindicative or may have unacceptably low factor loadings on every subscale, or completely missing subscales may complicate computations. In this article, it was studied how well multiple imputation recovered the results of several psychometrically important statistics in a data set with such properties. Analysis of this data set revealed that multiple imputation was able to recover the results of these analyses well. Also, a simulation study showed that multiple imputation produced small bias in these statistics for simulated data sets with the same properties.     

An Acquaintanceship Questionnaire as a Test of Sociability

Abstract

Symbolic racism and social dominance theories were compared by reanalysis of data from a national probability sample of 234 White Americans and by using observed-variables, structural equation models. Contrary to the conclusions reached by Jessor (1989), the results did not support the major contentions of symbolic racism theory; rather, they seemed more consistent with the assumptions of social dominance theory. The possibility that symbolic racism serves as an important legitimizing myth in American society is discussed.     

Identification of Differential Item Functioning in Multiple-Group Settings: A Multivariate Outlier Detection Approach

We focus on the identification of differential item functioning (DIF) when more than two groups of examinees are considered. We propose to consider items as elements of a multivariate space, where DIF items are outlying elements. Following this approach, the situation of multiple groups is a quite natural case. A robust statistics technique is proposed to identify DIF items as outliers in the multivariate space. For low dimensionalities, up to 2–3 groups, a simple graphical tool is derived. We illustrate our approach with a reanalysis of data from Kim, Cohen, and Park (1995) on using calculators for a mathematics test.     

Robust Mokken Scale Analysis by Means of the Forward Search Algorithm for Outlier Detection

Exploratory Mokken scale analysis (MSA) is a popular method for identifying scales from larger sets of items. As with any statistical method, in MSA the presence of outliers in the data may result in biased results and wrong conclusions. The forward search algorithm is a robust diagnostic method for outlier detection, which we adapt here to identify outliers in MSA. This adaptation involves choices with respect to the algorithm's objective function, selection of items from samples without outliers, and scalability criteria to be used in the forward search algorithm. The application of the adapted forward search algorithm for MSA is demonstrated using real data. Recommendations are given for its use in practical scale analysis.     

企业员工职业承诺的结构模型研究

在文献研究基础上,通过访谈等方法,编制职业承诺问卷(OCQ)。通过对240名被试进行预测,修订得到正式问卷。重新选取11家企业的员工进行测试,获得330份有效问卷,数据的验证性因素分析结果表明:企业员工的职业承诺是四因素结构,包括情感承诺、规范承诺、代价承诺和选择限制承诺。     

On the Treatment of Outliners in Cognitive and Psychomotor Test Data

Authors of many statistical texts and review articles have pointed to the possi- ble adverse effects that outliers can have on the calculation of sample statistics and have suggested several methods for detecting and treating outliers. We investigated two different methods-data censoring and transformation-for treating outliers in aptitude test data at the item level and total-score level and their effects on the internal consistency and predictive validity of six computer- ized tests being evaluated by the U.S. Air Force. Results from our sample of more than 2,000 pilot training candidates indicated that neither outlier treat- ment method at either level of analysis had significant effects on the tests' internal consistencies or predictive validities. Possible reasons for these findings include the frequency with which outliers occur and the robustness of linear modeling methods.     

Detection of Test Speededness Using Change-Point Analysis

Change-point analysis (CPA) is a well-established statistical method to detect abrupt changes, if any, in a sequence of data. In this paper, we propose a procedure based on CPA to detect test speededness. This procedure is not only able to classify examinees into speeded and non-speeded groups, but also identify the point at which an examinee starts to speed. Identification of the change point can be very useful. First, it informs decision makers of the appropriate length of a test. Second, by removing the speeded responses, instead of the entire response sequence of an examinee suspected of speededness, ability estimation can be improved. Simulation studies show that this procedure is efficient in detecting both speeded examinees and the speeding point. Ability estimation is dramatically improved by removing speeded responses identified by our procedure. The procedure is then applied to a real dataset for illustration purpose.     

Modeling Within-Item Dependencies in Parallel Data on Test Responses and Brain Activation

Psychometrika - In this paper, we propose a joint modeling approach to analyze dependency in parallel response data. We define two types of dependency: higher-level dependency and within-item...     

Anchoring and Adjusting in Questionnaire Responses

Do survey designers bias respondents' answers on attitude/opinion questionnaires through the organization of their survey items? We hypothesize that respondents often employ an anchoring and adjusting strategy in which their response to an initial survey item provides a cognitive anchor from which they insufficiently adjust in answering the subsequent item. Three experiments indicate that respondents often anchor and insufficiently adjust in certain situations. Ultimately, this tendency can affect reliability estimates of scales and the resultant correlations with other measures. In organizing their surveys, researchers may wish to combat this bias by intermixing items designed for different but related constructs.     

A Statistical Interpretive Method for Neuropsychological Test Data

The accurate interpretation of large numbers of neuropsychological tests within a flexible battery approach is a difficult and sometimes controversial process. We present a statistically based method of interpretation (Rohling's Interpretive Method or RIM) and evaluation of neuropsychological data that allows for varying numbers of tests along a varying number of cognitive domains, yet remains psychometrically based. This method requires informed clinical judgment in that the level of confidence for tests, cognitive domains, and global indices are used as the backdrop for interpretive decisions. Specific procedures for use are presented in a systematic, detailed fashion to allow the interested reader to replicate the method. Two case examples are presented: a straightforward case of cerebrovascular insult and a more complicated case of mixed etiology. Examples include a variety of different neuropsychological tests commonly used in a flexible battery approach. A discussion of the practicality, ease of use, and potential limitations of this method are further presented.     

Assessing the Multi-faceted Nature of Test Anxiety Among Secondary School Students: An English Version of the German Test Anxiety Questionnaire: PAF-E

The current study concerns the validation of an English version of the German Test Anxiety Inventory, namely the PAF-E. This questionnaire is a multi-faceted measure of test anxiety designed to detect normative test anxiety levels and in consequence meet the need of consultancy. Construct and criterion validity of (PAF-E) were examined with a sample of 96 secondary students (Mage = 12.8, SD = 0.67; 55% girls) from an international school in Berlin (Germany) and 399 secondary students (Mage = 13.4, SD = 0.80; 56% girls) from Montréal (Canada). Both samples completed the PAF-E and related constructs, such as school-related self-efficacy, inhibitory test anxiety, achievement motivation, and the Big Five. Exploratory and confirmatory factor analyses confirmed the four-factor-structure (worry, emotionality, interfering thoughts, lack of confidence) of the original German Test Anxiety Inventory (PAF). Each subscale consists of five items with a total of 20 questions. Cronbach's alpha, ranging from.71 to.82 among Germans and.77 to.87 among Canadians as well as the re-test reliability (from.80 to.85 among Canadians) were sufficient. The differential patterns of correlations between other constructs and the indices of test anxiety indicate good construct validity.     

感知/亲历医疗暴力医生的内向和外向反应——基于浙江调研数据的分析

自行设计量表并收集浙江省36家医院671名医生有效问卷,使用方差分析方法探讨感知/亲历医疗暴力的医生群体的内向和外向反应特征。研究发现,三级医院医生亲历医疗暴力比重显著高于基层医院。高医疗暴力发生率产生削弱医生工作积极性和子女学医支持度,并提升其高风险患者识别意愿的内向反应(P<0.05)。亲历医疗暴力医生感知的医疗暴力发生率高于未亲历暴力医生,且有更消极的内向反应和对于医疗暴力法律环境及医疗卫生改革政策更负面的外向反应(P<0.05)。接受反医疗暴力培训、了解暴力防控流程医生的内外向反应较为正面。     

Normative Data for the Boston Naming Test in Native Dutch-Speaking Belgian Elderly

A normative study of the 60-item version of the Boston Naming Test (BNT) was performed in a group of 200 native Dutch-speaking Flemish elderly. Analysis of test results revealed that BNT performance in Dutch is significantly affected by age, years of education, and gender. Error analysis disclosed verbal semantic paraphasias to occur as the most frequent error type (1/3 errors). “Don't know responses,” verbal semantic paraphasias, and adequate circumlocutions were found on at least 30 different BNT items and constituted the most diffusely distributed error types. Following a careful review of other normative BNT studies, group characteristics rather than cultural differences were found to account for the difference in the overall mean scores. Our study surprisingly revealed that, as far as American–English, Australian–English and Dutch-speaking elderly are concerned, linguistics do not have an impact on the overall mean BNT score. A linguistic impact, however, clearly holds on the qualitative levels of performance, reflected by fundamental differences in the error distribution in different languages. Language-related BNT characteristics therefore stress the need for specific adaptations of norms.     

Using Two-Level Factor Analysis to Test for Cluster Bias in Ordinal Data

The test for cluster bias is a test of measurement invariance across clusters in 2-level data. This article examines the true positive rates (empirical power) and false positive rates of the test for cluster bias using the likelihood ratio test (LRT) and the Wald test with ordinal data. A simulation study indicates that the scaled version of the LRT that accounts for nonnormality of the data gives untrustworthy results, whereas the unscaled LRT and the Wald test have acceptable false positive rates and perform well in terms of empirical power rate if the amount of cluster bias is large. The test for cluster bias is illustrated with data from research on teacher-student relations.