Model specification search using a genetic algorithm with factor reordering for a simple structure factor analysis model

Abstract:  Many techniques for automated model specification search based on numerical indices have been proposed, but no single decisive method has yet been determined. In the present article, the performance and features of the model specification search method using a genetic algorithm (GA) were verified. A GA is a robust and simple metaheuristic algorithm with great searching power. While there has already been some research applying metaheuristics to the model fitting task, we focus here on the search for a simple structure factor analysis model and propose a customized algorithm for dealing with certain problems specific to that situation. First, implementation of model specification search using a GA with factor reordering for a simple structure factor analysis is proposed. Then, through a simulation study using generated data with a known true structure and through example analysis using real data, the effectiveness and applicability of the proposed method were demonstrated.     

Stochastic search item selection for factor analytic models

In this paper we implement a Markov chain Monte Carlo algorithm based on the stochastic search variable selection method of George and McCulloch (1993) for identifying promising subsets of manifest variables (items) for factor analysis models. The suggested algorithm is constructed by embedding in the usual factor analysis model a normal mixture prior for the model loadings with latent indicators used to identify not only which manifest variables should be included in the model but also how each manifest variable is associated with each factor. We further extend the suggested algorithm to allow for factor selection. We also develop a detailed procedure for the specification of the prior parameters values based on the practical significance of factor loadings using ideas from the original work of George and McCulloch (1993). A straightforward Gibbs sampler is used to simulate from the joint posterior distribution of all unknown parameters and the subset of variables with the highest posterior probability is selected. The proposed method is illustrated using real and simulated data sets.     

Particle swarm optimised analysis of investment decision

A portfolio forecasting model based on particle swarm optimization (PSO) algorithm with automatic factor scaling is proposed in this Article to effectively improve the accuracy of related analysis model in portfolio application. Firstly, the portfolio problem is analyzed and a hybrid constraint portfolio model is obtained by improving portfolio model with consideration of general portfolio model and combination of market value constraint and upper bound constraint according to Markowitz's theory. Secondly, PSO algorithm is introduced during analysis on portfolio model and solution is found with the hybrid constraint portfolio model of PSO on portfolio. In addition, in order to further improve the performance of PSO in model solution, automatic factor scaling is used for adaptive learning on parameters associated with PSO, to improve convergence of the algorithm. At last, simulation experiments show that the algorithm proposed can obtain a more ideal investment portfolio scheme, thereby reducing investment risks and obtaining greater investment returns.     

Bayesian model comparison of nonlinear structural equation models with missing continuous and ordinal categorical data

Missing data are very common in behavioural and psychological research. In this paper, we develop a Bayesian approach in the context of a general nonlinear structural equation model with missing continuous and ordinal categorical data. In the development, the missing data are treated as latent quantities, and provision for the incompleteness of the data is made by a hybrid algorithm that combines the Gibbs sampler and the Metropolis‐Hastings algorithm. We show by means of a simulation study that the Bayesian estimates are accurate. A Bayesian model comparison procedure based on the Bayes factor and path sampling is proposed. The required observations from the posterior distribution for computing the Bayes factor are simulated by the hybrid algorithm in Bayesian estimation. Our simulation results indicate that the correct model is selected more frequently when the incomplete records are used in the analysis than when they are ignored. The methodology is further illustrated with a real data set from a study concerned with an AIDS preventative intervention for Filipina sex workers.     

Factor analysis for clustered observations

Classical factor analysis assumes a random sample of vectors of observations. For clustered vectors of observations, such as data for students from colleges, or individuals within households, it may be necessary to consider different within-group and between-group factor structures. Such a two-level model for factor analysis is defined, and formulas for a scoring algorithm for estimation with this model are derived. A simple noniterative method based on a decomposition of the total sums of squares and crossproducts is discussed. This method provides a suitable starting solution for the iterative algorithm, but it is also a very good approximation to the maximum likelihood solution. Extensions for higher levels of nesting are indicated. With judicious application of quasi-Newton methods, the amount of computation involved in the scoring algorithm is moderate even for complex problems; in particular, no inversion of matrices with large dimensions is involved. The methods are illustrated on two examples.Suggestions and corrections of three anonymous referees and of an Associate Editor are acknowledged. Discussions with Bob Jennrich on computational aspects were very helpful. Most of research leading to this paper was carried out while the first author was a visiting associate professor at the University of California, Los Angeles.     

A gauss-newton algorithm for exploratory factor analysis

It is shown that the scoring algorithm for maximum likelihood estimation in exploratory factor analysis can be developed in a way that is many times more efficient than a direct development based on information matrices and score vectors. The algorithm offers a simple alternative to current algorithms and when used in one-step mode provides the simplest and fastest method presently available for moving from consistent to efficient estimates. Perhaps of greater importance is its potential for extension to the confirmatory model. The algorithm is developed as a Gauss-Newton algorithm to facilitate its application to generalized least squares and to maximum likelihood estimation.This research was supported by NSF Grant MCS-8301587.     

Dynamic factor analysis of nonstationary multivariate time series

A dynamic factor model is proposed for the analysis of multivariate nonstationary time series in the time domain. The nonstationarity in the series is represented by a linear time dependent mean function. This mild form of nonstationarity is often relevant in analyzing socio-economic time series met in practice. Through the use of an extended version of Molenaar's stationary dynamic factor analysis method, the effect of nonstationarity on the latent factor series is incorporated in the dynamic nonstationary factor model (DNFM). It is shown that the estimation of the unknown parameters in this model can be easily carried out by reformulating the DNFM as a covariance structure model and adopting the ML algorithm proposed by Jöreskog. Furthermore, an empirical example is given to demonstrate the usefulness of the proposed DNFM and the analysis.     

Semi-sparse PCA

It is well known that the classical exploratory factor analysis (EFA) of data with more observations than variables has several types of indeterminacy. We study the factor indeterminacy and show some new aspects of this problem by considering EFA as a specific data matrix decomposition. We adopt a new approach to the EFA estimation and achieve a new characterization of the factor indeterminacy problem. A new alternative model is proposed, which gives determinate factors and can be seen as a semi-sparse principal component analysis (PCA). An alternating algorithm is developed, where in each step a Procrustes problem is solved. It is demonstrated that the new model/algorithm can act as a specific sparse PCA and as a low-rank-plus-sparse matrix decomposition. Numerical examples with several large data sets illustrate the versatility of the new model, and the performance and behaviour of its algorithmic implementation.

     

The nonconvergence of factals: A nonmetric common factor analysis

FACTALS is a nonmetric common factor analysis model for multivariate data whose variables may be nominal, ordinal or interval. In FACTALS an Alternating Least Squares algorithm is utilized which is claimed to be monotonically convergent.In this paper it is shown that this algorithm is based upon an erroneous assumption, namely that the least squares loss function (which is in this case a nonscale free loss function) can be transformed into a scalefree loss function. A consequence of this is that monotonical convergence of the algorithm can not be guaranteed.     

Probabilistic Disjoint Principal Component Analysis

One of the most relevant problems in principal component analysis and factor analysis is the interpretation of the components/factors. In this paper, disjoint principal component analysis model is extended in a maximum-likelihood framework to allow for inference on the model parameters. A coordinate ascent algorithm is proposed to estimate the model parameters. The performance of the methodology is evaluated on simulated and real data sets.     

Generalized latent variable models with non‐linear effects

Until recently, item response models such as the factor analysis model for metric responses, the two‐parameter logistic model for binary responses and the multinomial model for nominal responses considered only the main effects of latent variables without allowing for interaction or polynomial latent variable effects. However, non‐linear relationships among the latent variables might be necessary in real applications. Methods for fitting models with non‐linear latent terms have been developed mainly under the structural equation modelling approach. In this paper, we consider a latent variable model framework for mixed responses (metric and categorical) that allows inclusion of both non‐linear latent and covariate effects. The model parameters are estimated using full maximum likelihood based on a hybrid integration–maximization algorithm. Finally, a method for obtaining factor scores based on multiple imputation is proposed here for the non‐linear model.     

The factor structure of the Beck Depression Inventory-II: an evaluation

The Beck Depression Inventory-II (BDI-II) is a frequently used scale for measuring depressive severity. BDI-II data (404 clinical; 695 nonclinical adults) were analyzed by means of confirmatory factor analysis to test whether the factor structure model with a somatic-affective and cognitive component of depression, formulated by Beck and colleagues, has a good fit. We also evaluated 10 alternative models. The fit of Beck's model was not good for all criteria. Three of the alternative models had a better fit in both samples, but none of these met all criteria for good fit. Of the alternatives with a better fit, we selected the only model with unidimensional subscales, which assesses a somatic, affective, and cognitive dimension. For this model, which we recommend, as well as for Beck' original model, a good fitting structure containing 15 and 16 items was developed with an item-deletion algorithm.     

Selection of variables in exploratory factor analysis: An empirical comparison of a stepwise and traditional approach

The purpose of this study was to investigate and compare the performance of a stepwise variable selection algorithm to traditional exploratory factor analysis. The Monte Carlo study included six factors in the design; the number of common factors; the number of variables explained by the common factors; the magnitude of factor loadings; the number of variables not explained by the common factors; the type of anomaly evidenced by the poorly explained variables; and sample size. The performance of the methods was evaluated in terms of selection and pattern accuracy, and bias and root mean squared error of the structure coefficients. Results indicate that the stepwise algorithm was generally ineffective at excluding anomalous variables from the factor model. The poor selection accuracy of the stepwise approach suggests that it should be avoided.     

融合反应时的多级评分IRT模型开发及其应用研究

当前大多数融合反应时的IRT模型仅适用于0-1评分数据资料,极大的限制了IRT反应时模型在实际中的应用。本文在传统的二级计分反应时IRT模型基础上,拟开发一种多级评分反应时模型。在层次建模框架下,分别采用拓广分部评分模型(GPCM)和对数正态模型构建融合反应时的多级评分IRT模型(本文记为JRT-GPCM),并采用全息贝叶斯MCMC算法实现新模型的参数估计。为验证新开发的JRT-GPCM模型的可行性及其在实践中的应用,本文开展了两项研究:研究1为模拟实验研究,研究2为新模型在大五人格-神经质分量表中的应用。研究1结果表明,JRT-GPCM模型的估计精度较高,且具有较好的稳健性。研究2表明,被试的潜在特质与作答速度具有一定的正相关,且本研究结果支持Ferrando和Lorenzo-Seva(2007)提出的“距离-困难度假设”,即当被试的潜在特质与项目的难度阈限距离越远,那么被试会花费更多的时间对项目进行作答。总之,本研究为拓展反应时信息在心理测量及教育中的应用提供新的方法支持。     

Exploratory longitudinal factor analysis in multiple populations

For multiple populatios, a longtidinal factor analytic model which is entirely exploratory, that is, no explicit identification constraints, is proposed. Factorial collapse and period/practice effects are allowed. An invariant and/or stationary factor pattern is permitted. This model is formulated stochastically. To implement this model a stagewise EM algorithm is developed. Finally a numerical illustration utilizing Nesselroade and Baltes' data is presented.The authors wish to thank Barbara Mellers and Henry Kaiser for their helpful comments and John Nesselroade for providing us the data for our illustration. This research wwa supported in part by a grant (No. AG03164) from the National Institute on Aging to William Meredith. Details of the derivations and a copy of the PROC MATRIX program are available upon request from the first author.     

Three-Mode Component Analysis with Crisp or Fuzzy Partition of Units

A new methodology is proposed for the simultaneous reduction of units, variables, and occasions of a three-mode data set. Units are partitioned into a reduced number of classes, while, simultaneously, components for variables and occasions accounting for the largest common information for the classification are identified. The model is a constrained three-mode factor analysis and it can be seen as a generalization of the REDKM model proposed by De Soete and Carroll for two-mode data. The least squares fitting problem is mathematically formalized as a constrained problem in continuous and discrete variables. An iterative alternating least squares algorithm is proposed to give an efficient solution to this minimization problem in the crisp and fuzzy classification context. The performances of the proposed methodology are investigated by a simulation study comparing our model with other competing methodologies. Different procedures for starting the proposed algorithm have also been tested. A discussion of some interesting differences in the results follows. Finally, an application to real data illustrates the ability of the proposed model to provide substantive insights into the data complexities.     

EM algorithms for ML factor analysis

The details of EM algorithms for maximum likelihood factor analysis are presented for both the exploratory and confirmatory models. The algorithm is essentially the same for both cases and involves only simple least squares regression operations; the largest matrix inversion required is for aq ×q symmetric matrix whereq is the matrix of factors. The example that is used demonstrates that the likelihood for the factor analysis model may have multiple modes that are not simply rotations of each other; such behavior should concern users of maximum likelihood factor analysis and certainly should cast doubt on the general utility of second derivatives of the log likelihood as measures of precision of estimation.     

多维项目反应理论：参数估计及其在心理测验中的应用

本研究介绍并引进了现代测量理论中的前沿技术—— 多维项目反应理论, 采用MCMC算法实现了其参数估计; 并将MIRT应用于瑞文高级推理测验, 以探讨MIRT在心理测验中的具体应用。研究结果表明：(1)本研究自主编制的MIRT参数估计程序基本可行, 其估计的精度与国外研究结论相当甚至更好。(2)在测验维度和样本容量两因素完全随机实验设计下(2×3), 随着被试和题目样本容量的增加, MIRT参数估计的精度越高且估计的稳定性越强; 但随着测验维度的增加, MIRT参数估计精度和稳定性均随之降低。(3)MIRT对心理测验的分析比UIRT能提供更为精确和细致的信息。它对心理测验的编制、开发及评价具有重要的指导和参考价值, 值得引进及借鉴。     

Generating correlation matrices with model error for simulation studies in factor analysis: A combination of the Tucker-Koopman-Linn model and Wijsman’s algorithm

Most simulation studies in factor analysis follow a process of constructing population correlation matrices from the common-factor model and generating sample correlation matrices from the population matrices. In the common-factor model, the population correlation matrix is perfectly fit by the model’s containing common and unique factors. However, since no mathematical model accounts exactly for the real-world phenomena that it is intended to represent, the Tucker-Koopman-Linn model (1969) is more realistic for generating correlation matrices than the conventional common-factor model because the former incorporates model error. In this paper, a procedure for generating population and sample correlation matrices with model error by combining the Tucker-Koopman-Linn model and Wijsman’s algorithm (1959) is presented. The SAS/ IML program for generating correlation matrices is described, and an example is also provided.     

Influence analysis for the factor analysis model with ranking data

Influence analysis is an important component of data analysis, and the local influence approach has been widely applied to many statistical models to identify influential observations and assess minor model perturbations since the pioneering work of Cook (1986) . The approach is often adopted to develop influence analysis procedures for factor analysis models with ranking data. However, as this well‐known approach is based on the observed data likelihood, which involves multidimensional integrals, directly applying it to develop influence analysis procedures for the factor analysis models with ranking data is difficult. To address this difficulty, a Monte Carlo expectation and maximization algorithm (MCEM) is used to obtain the maximum‐likelihood estimate of the model parameters, and measures for influence analysis on the basis of the conditional expectation of the complete data log likelihood at the E‐step of the MCEM algorithm are then obtained. Very little additional computation is needed to compute the influence measures, because it is possible to make use of the by‐products of the estimation procedure. Influence measures that are based on several typical perturbation schemes are discussed in detail, and the proposed method is illustrated with two real examples and an artificial example.