Nonmetric multidimensional scaling with clustering of subjects

A new nonmetric multidimensional scaling method is devised to analyze three-way data concerning inter-stimulus similarities obtained from many subjects. It is assumed that subjects are classified into a small number of clusters and that the stimulus configuration is specific to each cluster. Under this assumption, the classification of subjects and the scaling used to derive the configurations for clusters are simultaneously performed using an alternating least-squares algorithm. The monotone regression of ordinal similarity data, the scaling of stimuli and the K -means clustering of subjects are iterated in the algorithm. The method is assessed using a simulation and its practical use is illustrated with the analysis of real data. Finally, some extensions are considered.     

Sequential estimation in multidimensional scaling

The concept of sequential estimation is introduced in multidimensional scaling (MDS). The sequential estimation method developed in this paper refers to continually updating estimates of a configuration as new observations are added. This method has a number of advantages, such as a locally optimal design of the experiment can be easily constructed, and dynamic experimentation is made possible. Using artificial data, the performance of our sequential method is illustrated.We are indebted to anonymous reviewers for their suggestions. In addition, we thank Dr. Frank Critchley for his helpful comments on our Q/S algorithm.     

Robust Mokken Scale Analysis by Means of the Forward Search Algorithm for Outlier Detection

Exploratory Mokken scale analysis (MSA) is a popular method for identifying scales from larger sets of items. As with any statistical method, in MSA the presence of outliers in the data may result in biased results and wrong conclusions. The forward search algorithm is a robust diagnostic method for outlier detection, which we adapt here to identify outliers in MSA. This adaptation involves choices with respect to the algorithm's objective function, selection of items from samples without outliers, and scalability criteria to be used in the forward search algorithm. The application of the adapted forward search algorithm for MSA is demonstrated using real data. Recommendations are given for its use in practical scale analysis.     

Latent Variable Selection for Multidimensional Item Response Theory Models via $$L_{1}$$ Regularization

We develop a latent variable selection method for multidimensional item response theory models. The proposed method identifies latent traits probed by items of a multidimensional test. Its basic strategy is to impose an \(L_{1}\) penalty term to the log-likelihood. The computation is carried out by the expectation–maximization algorithm combined with the coordinate descent algorithm. Simulation studies show that the resulting estimator provides an effective way in correctly identifying the latent structures. The method is applied to a real dataset involving the Eysenck Personality Questionnaire.     

Distance stability analysis in multidimensional scaling using the jackknife method

Stability or sensitivity analysis is an important topic in data analysis that has received little attention in the application of multidimensional scaling (MDS), for which the only available approaches are given in terms of a coordinate‐based analytical jackknife methodology. Although in MDS the prime interest is in assessing the stability of the points in the configuration, this methodology may be influenced by imprecisions resulting from the inherently necessary Procrustes method. This paper proposes an analytical distance‐based jackknife procedure to study stability and cross‐validation in MDS in terms of the jackknife distances, which is not influenced by the Procrustes method. For each object, the corresponding jackknife estimated points are considered as naturally clustered points, and stability and cross‐validation are analysed in terms of the MDS distances arising from the jackknife procedure, on the basis of a weighted cluster‐MDS algorithm. A jackknife‐relevant configuration is also proposed for cross‐validation in terms of coordinates, in a cluster‐MDS framework.     

Latent variable selection in multidimensional item response theory models using the expectation model selection algorithm

The aim of latent variable selection in multidimensional item response theory (MIRT) models is to identify latent traits probed by test items of a multidimensional test. In this paper the expectation model selection (EMS) algorithm proposed by Jiang et al. (2015) is applied to minimize the Bayesian information criterion (BIC) for latent variable selection in MIRT models with a known number of latent traits. Under mild assumptions, we prove the numerical convergence of the EMS algorithm for model selection by minimizing the BIC of observed data in the presence of missing data. For the identification of MIRT models, we assume that the variances of all latent traits are unity and each latent trait has an item that is only related to it. Under this identifiability assumption, the convergence of the EMS algorithm for latent variable selection in the multidimensional two-parameter logistic (M2PL) models can be verified. We give an efficient implementation of the EMS for the M2PL models. Simulation studies show that the EMS outperforms the EM-based L₁ regularization in terms of correctly selected latent variables and computation time. The EMS algorithm is applied to a real data set related to the Eysenck Personality Questionnaire.     

Multidimensional modeling with unidimensional approximations

This paper advances nonparametric multidimensional item response theory by reporting experimental results on the use of nonmetric multidimensional scaling (MDS) to synthesize a multidimensional model from several approximating one-dimensional models. A two-dimensional simulation data set contains items in which the two-component traits combine linearly (dominance model items) and items in which the two-component traits combine quadratically (ideal point items). Several unidimensional approximations of the two-dimensional model were obtained by running unidimensional estimation software on the simulated data set. The graphs reconstructed from MDS of the unidimensional approximations at selected points clearly separate dominance items from ideal point items, and also various types of dominance or ideal point models. MDS also succeeded in determining the dimensionality of the simulation model items from the observable item responses.     

The item structure of the Eysenck Personality Questionnaire across method and culture

Since its publication, the Eysenck Personality Questionnaire (EPQ) has endured much criticism of its psychometric properties. As the most stringent of these attacks has derived from hierarchical factor analytic studies it is reasonable to ask whether the inadequacies reported accrue to the questionnaire itself or to the problematic features of factor analysis as a method. A further feature of empirical studies that are critical of the EPQ is that they are, by-and-large, based upon non-British samples. As the EPQ was constructed in Britain it is quite conceivable that cultural artifacts are responsible for the inconsistent findings of Canadian and American researchers. This paper offers a brief review of some of the critical research and reports a study in which the item structure of the EPQ is explored by non-metric multidimensional scaling techniques upon a sample of Irish adolescents. The MDS configuration was then compared with the higher order factor analytic results of American researchers to examine invariance across method. A clear item structure emerges supporting the Eysencks' theoretical model although items of the P scale are less homogenous than would be ideal for a psychometric tool.     

The Assessment of Dimensionality for Use in Item Response Theory

The application of item response theory (IRT) models requires the identification of the data's dimensionality. A popular method for determining the number of latent dimensions is the factor analysis of a correlation matrix. Unlike factor analysis, which is based on a linear model, IRT assumes a nonlinear relationship between item performance and ability. Because multidimensional scaling (MDS) assumes a monotonic relationship this method may be useful for the assessment of a data set's dimensionality for use with IRT models. This study compared MDS, exploratory and confirmatory factor analysis (EFA and CFA, respectively) in the assessment of the dimensionality of data sets which had been generated to be either one- or two-dimensional. In addition, the data sets differed in the degree of interdimensional correlation and in the number of items defining a dimension. Results showed that MDS and CFA were able to correctly identify the number of latent dimensions for all data sets. In general, EFA was able to correctly identify the data's dimensionality, except for data whose interdimensional correlation was high.     

On-line multidimensional scaling programs with interactive graphical display

The beginnings of a system of interactive multidimensional scaling programs with real-time display of the graphical output have been established on the Honeywell DDP-224 computer. Two programs have been completed: (1) MDPREF—a computer program for multidimensional analysis of preference data—has been converted from the GE-635 to run interactively on the DDP-224 computer. Its solution is printed from a typewriter, and the configuration of stimuli are displayed on a scope in two-dimensional view. (2) ROTATE—an on-line rotation program—enables the user to rotate the configuration in three dimensions within a higher dimensional space.     

MULTIDIMENSIONAL SCALING WITH A PRIORI DIMENSIONS

W aern Y. Multidimensional scaling with a priori dimensions. Scand. J. Psychol., 1972, 13 , 178–189.—This paper presents a method of multidimensional scaling by direct similarity estimates. The method is presented as an alternative to other multidimensional scaling methods, when hypotheses can be made concerning the dimensions underlying the perceptual variation. The method is used with five different sets of stimuli and compared, to an as to dimensions assumption-free component analysis, based on a vector model. The two methods resulted in similar configurations in most cases. One difference was found in a colour material, where the assumed dimensions could be regarded not to be orthogonal to each other. Here the a priori scaling revealed the assumed configuration as projected on the oblique axes whereas the component analysis could not give this information.     

Perhaps Unidimensional Is Not Unidimensional

Miller (1956) identified his famous limit of 7 ± 2 items based in part on absolute identification—the ability to identify stimuli that differ on a single physical dimension, such as lines of different length. An important aspect of this limit is its independence from perceptual effects and its application across all stimulus types. Recent research, however, has identified several exceptions. We investigate an explanation for these results that reconciles them with Miller’s work. We find support for the hypothesis that the exceptional stimulus types have more complex psychological representations, which can therefore support better identification. Our investigation uses data sets with thousands of observations for each participant, which allows the application of a new technique for identifying psychological representations: the structural forms algorithm of Kemp and Tenenbaum (2008) . This algorithm supports inferences not possible with previous techniques, such as multidimensional scaling.     

Choice model analysis of the “pick any/n” type of binary data

In the “pick any/n” method, subjects are asked to choose any number of items from a list of n items according to some criterion. This kind of data can be analyzed as a special case of either multiple-choice data or successive categories data where the number of response categories is limited to two. An item response model was proposed for the latter case, which is a combination of an unfolding model and a choice model. The marginal maximum-likelihood estimation method was developed for parameter estimation to avoid incidental parameters, and an expectation-maximization algorithm used for numerical optimization. Two examples of analysis are given to illustrate the proposed method, which we call MAXSC.     

Subjective geographic distance: A multidimensional comparison

The interdistances between thirteen places situated in different parts of the world were estimated by 60 subjects. The estimates were analysed by Kruskal's multidimensional technique and, after a cosine transformation, by factor analysis. It was found that both methods yielded the same three-dimensional solution. Also a two-dimensional configuration could describe the data, and it was shown that this configuration contained representations of unidimensional ratings obtained in other studies. This investigation was supported by grants from the Bank of Sweden Tercentenary Fund and the Swedish Council for Social Science Research. Died 5 May 1971.     

Indexes of Item-Objective Congruence for Multidimensional Items

The index of item-objective congruence developed by Rovinelli and Hambleton (1977) is a procedure used in test development for evaluating content validity at the item development stage. This measure is limited to the assessment of unidimensional items or items that measure specified composites of skills. In modern test development, items are sometimes developed to be multidimensional assessments or measures of multiple combinations of skills. The purpose of this research is to provide a mathematical extension to the Rovinelli and Hambleton index that is applicable for the multidimensional case.     

计算机化多维测验中作答时间和作答精度数据的联合分析

随着心理与教育测量研究的发展和科技的进步,计算机化(大规模)测验逐渐受到人们的关注。为探究在计算机化多维测验中如何利用作答时间数据来辅助评估多维潜在能力,以及为我国义务教育阶段教育质量监测提供数据分析方法上的理论支持。本研究以2012年和2015年国际学生能力评估(PISA)计算机化数学测验数据为例,提出了一种可同时利用作答时间和作答精度数据的联合作答与时间的多维Rasch模型。根据新模型对PISA数据的分析结果,表明引入作答时间数据,不仅有助于提高模型参数的估计精度,还有助于数据分析者利用被试的作答时间信息来做进一步的决策和干预(e.g., 对异常作答行为或预备知识的诊断)。     

Scale Alignment in the Between-Item Multidimensional Partial Credit Model

In between-item multidimensional item response models, it is often desirable to compare individual latent trait estimates across dimensions. These comparisons are only justified if the model dimensions are scaled relative to each other. Traditionally, this scaling is done using approaches such as standardization—fixing the latent mean and standard deviation to 0 and 1 for all dimensions. However, approaches such as standardization do not guarantee that Rasch model properties hold across dimensions. Specifically, for between-item multidimensional Rasch family models, the unique ordering of items holds within dimensions, but not across dimensions. Previously, Feuerstahler and Wilson described the concept of scale alignment, which aims to enforce the unique ordering of items across dimensions by linearly transforming item parameters within dimensions. In this article, we extend the concept of scale alignment to the between-item multidimensional partial credit model and to models fit using incomplete data. We illustrate this method in the context of the Kindergarten Individual Development Survey (KIDS), a multidimensional survey of kindergarten readiness used in the state of Illinois. We also present simulation results that demonstrate the effectiveness of scale alignment in the context of polytomous item response models and missing data.     

反应风格的测量与统计控制

反应风格是共同方法偏差的主要来源之一。本文首先讨论反应风格的定义和类型,梳理其危害,认为反应风格能使测验分数出现偏差,影响测验信效度分析和变量关系分析,有必要控制其危害。然后介绍了常用的反应风格测量方法,包括计数法和模型法两大类,对测量方法的选择给出了建议,在此基础上,就如何结合反应风格的测量方法与残差回归法、偏相关法来控制反应风格危害给出建议。     

反应风格的测量与统计控制

反应风格是共同方法偏差的主要来源之一。本文首先讨论反应风格的定义和类型,梳理其危害,认为反应风格能使测验分数出现偏差,影响测验信效度分析和变量关系分析,有必要控制其危害。然后介绍了常用的反应风格测量方法,包括计数法和模型法两大类,对测量方法的选择给出了建议,在此基础上,就如何结合反应风格的测量方法与残差回归法、偏相关法来控制反应风格危害给出建议。