Some Characteristics Of The G Index Of Agreement

Several uses of the G index of agreement are discussed for both Q- and R-techniques. It is shown that the index is meaningful and is not sensitive either to changes in having direction (for profile analysis) or to marginal inequalities for dichotomous data.     

Comparing continuous and dichotomous scoring of the balanced inventory of desirable responding

The Balanced Inventory of Desirable Responding (BIDR; Paulhus, 1994) is a widely used instrument to measure the 2 components of social desirability: self-deceptive enhancement and impression management. With respect to scoring of the BIDR, Paulhus (1994) authorized 2 methods, namely continuous scoring (all answers on the continuous answer scale are counted) and dichotomous scoring (only extreme answers are counted). In this article, we report 3 studies with student samples, and continuous and dichotomous scoring of BIDR subscales are compared with respect to reliability, convergent validity, sensitivity to instructional variations, and correlations with personality. Across studies, the scores from continuous scoring (continuous scores) showed higher Cronbach's alphas than those from dichotomous scoring (dichotomous scores). Moreover, continuous scores showed higher convergent correlations with other measures of social desirability and more consistent effects with self-presentation instructions (fake-good vs. fake-bad instructions). Finally, continuous self-deceptive enhancement scores showed higher correlations with those traits of the Five-factor model for which substantial correlations were expected (i.e., Neuroticism, Extraversion, and Conscientiousness). Consequently, these findings indicate that continuous scoring may be preferable to dichotomous scoring when assessing socially desirable responding with the BIDR.     

Early maladaptive schema and social and occupational functioning: The mediating role of psychiatric symptomatology

Early Maladaptive Schemas (EMS) are described as long‐held core beliefs which are dysfunctional to a significant degree. However, the supposition that schemas are dysfunctional by nature, while not isomorphic with psychiatric syndromes, is yet to be subjected to empirical review. The current study seeks to investigate the relationship between the Young Schema Questionnaire and the concept of ‘dysfunction’ in a community sample to determine the indirect effects of psychiatric symptomatology and validate current scoring guidelines with a convergent measure of dysfunction. A total of 464 people completed a survey online comprising of the YSQ‐Short Form, the Depression, Anxiety and Stress Scales‐21, the Social and Occupational Functioning Assessment Scales, and the World Health Organization Quality of Life Scale. Multiple regression analyses revealed a moderate relationship between EMS categories and measures of dysfunction, however only six of eighteen EMS categories were significant predictors in this model. Mediation analyses further suggest that the relationship between EMS and dysfunction is partially mediated by psychiatric symptomatology. The current dichotomous clinical scoring guidelines were found to be invalid when measures of functioning were used as convergent measures for twelve of the EMS categories. These findings suggest the YSQ is best conceptualised as a general measure of schema as opposed to a measure of EMS categories.     

Identification,Reflection, And Correlation: Problems In The Bases Of Repertory Grid Measures

Repertory grids, deriving from George Kelly's personal construct theory, have been used to provide measures of a number of personality and cognitive variables. Several of these grid measures, such as the identification index, some measures of cognitive complexity, and other indices extracted from factor analyses of grids, are based on correlations between the columns (elements) of the grid data matrix. These measures are problematic and unstable because the intercolumn correlations depend on the direction of scoring across each of the matrix rows (constructs). This direction is not guided by explicit or theoretically justified rules and appears to be arbitrary and inconsistent between researchers. Also, correlation is a poor measure of element similarity, the basis of the identification measure. The importance of the valuating aspect of construing may provide a basis for the standardization of scoring. And scoring from the valued pole of a construct may help bring stability and meaning to the correlation-based measures.     

A Nonparametric Coefficient Of Internal Consistency

This paper discusses a frequency based, nonparametric measure of internal test consistency, referred to herein as coefficient alpha_τ, which allows facile measurement of the significance of differences in internal consistency between tests, administrations, or scoring methods. It also permits analysis of psychological tests containing items with discrete categories of response, yielding nominal scale data. Use of alpha_τ encourages flexibility in test construction, since multiple dimensions can be incorporated into individual test items.     

Psychometric Evaluation of an Alternate Scoring for the Remote Associates Test

The Remote Associates Test (RAT; Mednick, 1962; Mednick & Mednick, 1967) is a commonly employed test of creative convergent thinking. The RAT is scored with a dichotomous scoring, scoring correct answers as 1 and all other answers as 0. Based on recent research into the information processing underlying RAT performance, we argued that the dichotomous scoring may lead to a loss of potentially relevant information. Thus, we proposed an alternate scoring based on semantic similarity between the answer given by the participant and the correct solution using Latent Semantic Analysis (LSA; Landauer & Dumais, 1997). We evaluate the psychometric properties of the alternate LSA scoring and found evidence of construct validity for the LSA scoring which was comparable to findings for the standard scoring, but not better as we would have expected. Thus, our expectations that LSA-based scoring of the RAT counteracts potential information loss were not met. However, LSA based scorings appear to be a promising alternative for hardly solvable RAT items. We conducted additional analyses comparing different RAT item types with regard to their validity as well as evaluating the information uniquely contained in the LSA scoring. Implications of all finding for existing research using RAT items are discussed.     

Construct validity of the medical specialty preference inventory: A critical analysis

The Medical Specialty Preference Inventory (MSPI; Zimny, G.H. (1979). Manual for the Medical Specialty Preference Inventory. St. Louis, MO: St. Louis University School of Medicine), a measure of medical students’ interests, was substantively and empirically examined to identify an underlying factor structure. A factor model for the original MSPI based on 38 factors in five general areas was evaluated on a national sample of 1014 medical students and yielded poor fit to the data. Exploratory factor analyses at the item level utilizing the full pool of MSPI items produced an 11 factor solution with 88 items. Sub-scales were identified within this model and an 11-18 higher-order model and an 18 sub-scale model also were proposed. The relative fits of the three models were evaluated by confirmatory factor analysis with the 18 sub-scale model shown to be superior. This model was cross-validated on a separate sample of 1016 medical students and fit the data well. All sub-scales exhibited adequate internal consistency across samples. These findings support the need for a revised MSPI based on 18 scales. Implications of these findings for MSPI scoring practices are discussed along with future directions.     

重参数化多分属性DINA模型的多级评分拓广——基于等级反应模型

多分属性认知诊断模型（CDMs）比传统的二分属性CDMs提供更详细的诊断反馈信息,但现有大部分多分属性CDMs并不具备直接分析多级（或混合）评分数据的功能。本文基于等级反应模型对重参数化多分属性DINA模型进行多级评分拓广,开发一个可处理多级评分数据的等级反应多分属性DINA模型。首先通过实证数据分析呈现新模型的现实可应用性;然后通过模拟研究探究新模型的参数估计返真性。结果表明,新模型满足同时处理多分属性和多级评分数据的现实需求;且具备良好的心理计量学性能,但对测验质量有一定要求（如题目质量较高且测验Qp矩阵具有完备性等）。     

Estimating item parameters and latent ability when responses are scored in two or more nominal categories

A multivariate logistic latent trait model for items scored in two or more nominal categories is proposed. Statistical methods based on the model provide 1) estimation of two item parameters for each response alternative of each multiple choice item and 2) recovery of information from wrong responses when estimating latent ability. An application to a large sample of data for twenty vocabulary items shows excellent fit of the model according to a chi-square criterion. Item and test information curves are compared for estimation of ability assuming multiple category and dichotomous scoring of these items. Multiple scoring proves substantially more precise for subjects of less than median ability, and about equally precise for subjects above the median.Preparation of this paper was supported in part by N.S.F. Grant GS-2900.     

拓广分部评分模型下计算机自适应测验变加权选题策略

摘要: 在计算机自适应测验中, 对0-1评分模型按a-分层选题是高效安全的策略,但多级评分模型的项目难度/步骤参数有多个而无法直接应用这种选题策略。信息函数能够很好地综合项目所有参数及能力参数,但最大信息量选题策略会影响考试安全。本文提出一种变加权选题策略,它通过调用一个与信息量相关联的函数,该函数与信息量成正比,与区分度的某个幂函数成反比,从而达到既能综合项目所有参数又按a分层的效果。在GPCM模型下用蒙特卡罗实验进行比较研究,结果显示新的选题策略总体效果比已有相关结果好。     

A longitudinal study of bilingual and cognitive development(1)

This paper presents the findings of a longitudinal study of IQ data collected over a 5-year period (Grades K-4) on pupils enrolled in a French immersion program (anglophone pupils receiving all instruction in French except English language arts) and pupils in the regular English program. Although year-by-year results may fail to show IQ differences between the two groups, repeated measures analysis indicates that the immersion group has a higher IQ measure over the 5-year period. However, considering Grades 1–3 only, which involved the administration of the same test, the two groups do not score differentially with respect to either overall IQ measure or specific subtest scores (classification/categorization, analogies, following of verbal directions) when scores are adjusted for initial IQ (and age) differences, thus failing initially to support studies which show positive relationships between bilingualism and cognitive functioning. However, supportive of those studies is a further analysis on the data of immersion pupils classified as “high” French achievers vs. “low” French achievers. The high French achievers obtain significantly higher IQ measures and subtest scores (analogies and following of verbal directions) than the low French achievers, even when scores are adjusted for initial IQ and age differences. These findings are interpreted in the context of Cummins' (1976) “threshold” hypothesis relating to level of competence in the second language.     

Validity of the German Test Anxiety Inventory (TAI‐G) in an Australian sample

Test anxiety (TA) is a prevalent issue among students that can result in deleterious consequences, such as underachievement. However, a contemporary measure that has been validated for use with Australian students seems to be lacking. This study, therefore, investigated the suitability of the German Test Anxiety Inventory (TAI‐G) for use with Australian university students. While the original TAI‐G contains 30 items and was designed to measure four factors (worry, emotionality, interference, and lack of confidence), differing factorial models have been supported in the literature using either the original or a shortened 17‐item version of the measure. These differing TAI‐G models were tested and compared in the current study via confirmatory factor analysis using 224 Australian university students. As expected, results supported the superior fit of the 17‐item four‐factor model. Additionally, the convergent validity of the measure was supported since measures of self‐esteem, self‐efficacy, and general anxiety were all found to correlate significantly with the TAI‐G in the hypothesised directions. Finally, the finding that all of the TAI‐G subscales had acceptably high reliabilities led to the conclusion that the 17‐item TAI‐G is a valid and reliable measure of TA in an Australian university population.     

A marginal maximum likelihood method for the vector threshold model to analyze dichotomous choice data

Abstract: At least two types of models, the vector model and the unfolding model can be used for the analysis of dichotomous choice data taken from, for example, the pick any/ n method. The previous vector threshold models have a difficulty with estimation of the nuisance parameters such as the individual vectors and thresholds. This paper proposes a new probabilistic vector threshold model, where, unlike the former vector models, the angle that defines an individual vector is a random variable, and where the marginal maximum likelihood estimation method using the expectation-maximization algorithm is adopted to avoid incidental parameters. The paper also attempts to discuss which of the two models is more appropriate to account for dichotomous choice data. Two sets of dichotomous choice data are analyzed by the model.     

Alternate methods and software for calculating interobserver agreement for continuous observation data

Portable electronic data collection devices permit investigators to collect large amounts of observational data in a form ready for computer analysis. These devices are particularly efficient for gathering continuous data on multiple behavior categories. We expect that the increasing availability of these devices will lead to greater use of continuous data collection methods in observational research. This paper addresses the difficulties encountered when calculating traditional interobserver agreement statistics for continuous, multiple-code scoring. Two alternative strategies are described that yield interobserver agreement values based on the exact time of behavior code entries by the primary and secondary observers.Work on this paper was supported in part by NICHD Grants P01HD15051 and R01HD17650 and Office of Special Education and Rehabilitation Services Grant G008302980.     

A Comparison of Imputation Strategies for Ordinal Missing Data on Likert Scale Variables

This article compares a variety of imputation strategies for ordinal missing data on Likert scale variables (number of categories = 2, 3, 5, or 7) in recovering reliability coefficients, mean scale scores, and regression coefficients of predicting one scale score from another. The examined strategies include imputing using normal data models with naïve rounding/without rounding, using latent variable models, and using categorical data models such as discriminant analysis and binary logistic regression (for dichotomous data only), multinomial and proportional odds logistic regression (for polytomous data only). The result suggests that both the normal model approach without rounding and the latent variable model approach perform well for either dichotomous or polytomous data regardless of sample size, missing data proportion, and asymmetry of item distributions. The discriminant analysis approach also performs well for dichotomous data. Naïvely rounding normal imputations or using logistic regression models to impute ordinal data are not recommended as they can potentially lead to substantial bias in all or some of the parameters.     

An extension of the rating scale model with an application to the measurement of change

The polytomous unidimensional Rasch model with equidistant scoring, also known as the rating scale model, is extended in such a way that the item parameters are linearly decomposed into certain basic parameters. The extended model is denoted as the linear rating scale model (LRSM). A conditional maximum likelihood estimation procedure and a likelihood-ratio test of hypotheses within the framework of the LRSM are presented. Since the LRSM is a generalization of both the dichotomous Rasch model and the rating scale model, the present algorithm is suited for conditional maximum likelihood estimation in these submodels as well. The practicality of the conditional method is demonstrated by means of a dichotomous Rasch example with 100 items, of a rating scale example with 30 items and 5 categories, and in the light of an empirical application to the measurement of treatment effects in a clinical study.Work supported in part by the Fonds zur Förderung der Wissenschaftlichen Forschung under Grant No. P6414.     

On the generalization of the G index of agreement, Go, for use with ordinal scores

Abstract.— Presents a G index generalization for items where each has a predetermined number of response alternatives on an ordinal level. Shows that G_o is equal to the G index based on another set of dichotomous data. The problem that items with many response levels will be given more weight than those with few response levels is pointed out. An alternative index, G _0e, which does not have such characteristics, is presented and also proves to be equal to the G index based on a set of equivalent dichotomies.     

Stochastic Ordering Of the Latent Trait by the Sum Score Under Various Polytomous IRT Models

The sum score is often used to order respondents on the latent trait measured by the test. Therefore, it is desirable that under the chosen model the sum score stochastically orders the latent trait. It is known that unlike dichotomous item response theory (IRT) models, most polytomous IRT models do not imply stochastic ordering. It is unknown, however, (1) whether stochastic ordering is often or rarely violated and (2) whether violations yield a serious problem for practical data analysis. These are the central issues of this paper. First, some unanswered questions that pertain to polytomous IRT models implying stochastic ordering were investigated. Second, simulation studies were conducted to evaluate stochastic ordering in practical situations. It was found that for most polytomous IRT models that do not imply stochastic ordering, the sum score can be used safely to order respondents on the latent trait.The author would like to thank Klaas Sijtsma for commenting on earlier drafts of this paper.     

Discriminant Analysis Using Mixed Continuous,Dichotomous, and Ordered Categorical Variables

This article proposes an intuitive approach for predictive discriminant analysis with mixed continuous, dichotomous, and ordered categorical variables that are defined via an underlying multivariate normal distribution with a threshold specification. The classification rule is based on the comparison of the observed data logarithm probability density functions. To reduce the computational burden, the analysis is conducted in the context of a confirmatory factor analysis model with independent error measurements. Identification of the dichotomous and ordered categorical variables is discussed. Results are obtained by implementations of a Monte Carlo expectation maximization (MCEM)algorithm and a path sampling procedure. Probabilities of misclassification are estimated via the idea of the “jackknife” method. A real example is given to illustrate the proposed method.