The EM algorithm for latent class analysis with equality constraints

The EM algorithm is a popular iterative method for estimating parameters in the latent class model where at each step the unknown parameters can be estimated simply as weighted sums of some latent proportions. The algorithm may also be used when some parameters are constrained to equal given constants or each other. It is shown that in the general case with equality constraints, the EM algorithm is not simple to apply because a nonlinear equation has to be solved. This problem arises, mainly, when equality constrints are defined over probabilities indifferent combinations of variables and latent classes. A simple condition is given in which, although probabilities in different variable-latent class combinations are constrained to be equal, the EM algorithm is still simple to apply.The authors are grateful to the Editor and the anonymous reviewers for their helpful comments on an earlier draft of this paper. C. C. Clogg and R. Luijkx are also acknowledged for verifying our results with their computer programs MLLSA and LCAG, respectively.     

Component analysis with different sets of constraints on different dimensions

Many of the classical multivariate data analysis and multidimensional scaling techniques call for approximations by lower dimensional configurations. A model is proposed, in which different sets of linear constraints are imposed on different dimensions in component analysis and classical multidimensional scaling frameworks. A simple, efficient, and monotonically convergent algorithm is presented for fitting the model to the data by least squares. The basic algorithm is extended to cover across-dimension constraints imposed in addition to the dimensionwise constraints, and to the case of a symmetric data matrix. Examples are given to demonstrate the use of the method.The work reported in this paper has been supported by the Natural Sciences and Engineering Research Council of Canada, grant number A6394, and by the McGill-IBM Cooperative Grant, both granted to the first author. The research of H. A. L. Kiers has been made possible by a fellowship of the Royal Netherlands Academy of Arts and Sciences. We thank Michael Hunter for his helpful comments on earlier drafts of this paper.     

多项式加工树模型次序约束的量化分析及其应用

多项式加工树（MPT）模型是一种认知测量模型,能够对潜在认知过程进行测量和检验。已有研究探讨了二链MPT模型次序约束的重新参数化问题,本研究探讨了MPT模型次序约束的量化分析方法并从二链推广到多链,同时归纳出MPT模型参数向量内和参数向量间两参数次序约束量化分析的结论。数据分析结果表明该方法不仅在MPT模型框架下验证了潜在参数次序关系,而且给出了约束的量化指标,为潜在认知测量提供更有意义的解释。     

Constrained maximum likelihood estimation of two-level covariance structure model via EM type algorithms

In this paper, the constrained maximum likelihood estimation of a two-level covariance structure model with unbalanced designs is considered. The two-level model is reformulated as a single-level model by treating the group level latent random vectors as hypothetical missing-data. Then, the popular EM algorithm is extended to obtain the constrained maximum likelihood estimates. For general nonlinear constraints, the multiplier method is used at theM-step to find the constrained minimum of the conditional expectation. An accelerated EM gradient procedure is derived to handle linear constraints. The empirical performance of the proposed EM type algorithms is illustrated by some artifical and real examples.This research was supported by a Hong Kong UCG Earmarked Grant, CUHK 4026/97H. We are greatly indebted to D.E. Morisky and J.A. Stein for the use of their AIDS data in our example. We also thank the Editor, two anonymous reviewers, W.Y. Poon and H.T. Zhu for constructive suggestions and comments in improving the paper. The assistance of Michael K.H. Leung and Esther L.S. Tam is gratefully acknowledged.     

A new algorithm for the least-squares solution in factor analysis

A new algorithm to obtain the least-squares or MINRES solution in common factor analysis is presented. It is based on the up-and-down Marquardt algorithm developed by the present authors for a general nonlinear least-squares problem. Experiments with some numerical models and some empirical data sets showed that the algorithm worked nicely and that SMC (Squared Multiple Correlation) performed best among four sets of initial values for common variances but that the solution might sometimes be very sensitive to fluctuations in the sample covariance matrix.Numerical computation was made on a NEAC S-1000 computer in the Computer Center, Osaka University.     

Bayesian Estimation of Circumplex Models Subject to Prior Theory Constraints and Scale-Usage Bias

This paper presents a hierarchical Bayes circumplex model for ordinal ratings data. The circumplex model was proposed to represent the circular ordering of items in psychological testing by imposing inequalities on the correlations of the items. We provide a specification of the circumplex, propose identifying constraints and conjugate priors for the angular parameters, and accommodate theory-driven constraints in the form of inequalities. We investigate the performance of the proposed MCMC algorithm and apply the model to the analysis of value priorities data obtained from a representative sample of Dutch citizens. We wish to thank Michael Browne and two anonymous reviewers for their comments. The data for this study were collected as part of the project AIR2-CT94-1066, sponsored by the European Commission.     

Generalized constrained multiple correspondence analysis

A comprehensive approach for imposing both row and column constraints on multivariate discrete data is proposed that may be called generalized constrained multiple correspondence analysis (GCMCA). In this method each set of discrete data is first decomposed into several submatrices according to its row and column constraints, and then multiple correspondence analysis (MCA) is applied to the decomposed submatrices to explore relationships among them. This method subsumes existing constrained and unconstrained MCA methods as special cases and also generalizes various kinds of linearly constrained correspondence analysis methods. An example is given to illustrate the proposed method.Heungsun Hwang is now at Claes Fornell International Group. The work reported in this paper was supported by Grant A6394 from the Natural Sciences and Engineering Research Council of Canada to the second author.     

A Multiobjective Diet Planning Support System Using the Satisficing Trade-off Method

This paper shows that the satisficing trade-off method (STOM), one of the interactive multiobjective programming techniques, can be applied effectively to the formulation of livestock rations. Nutrient requirements are considered as soft constraints whose right-hand sides are flexible to some extent. This is easily done by regarding the values of the right-hand sides of the constraints as aspiration levels of the decision maker for objective functions. In this way a well-balanced solution can be obtained by STOM with automatic trade-off analysis. In STOM, moreover, because the objective functions and the constraint functions are interchangeable, decision makers are not required to consider the role of objective and constraint as fixed from the beginning. The authors’ experience is that the method makes ration formulation very easy, rapid and flexible. In addition, this paper shows how effectively STOM can be applied not only to diet planning but also to nutritional diagnosis. © 1997 by John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 6 : 131–139 (1997) No. of Figures: 1. No. of Tables: 3. No. of References: 28.     

Parameterizing inequality constraints on unique variances in linear structural models

Current computer programs for analyzing linear structural models will apparently handle only two types of constraints: fixed parameters, and equality of parameters. An important constraint not handled is inequality; this is particularly crucial for preventing negative variance estimates. In this paper, a method is described for imposing several kinds of inequality constraints in models, without the necessity for having computer programs which explicitly allow such constraints. The examples discussed include the prevention of Heywood cases, extension to inequalities of parameters to be greater than a specified value, and imposing ordered inequalities. Work on this project was aided by the City University of New York—Professional Staff Congress Research Award Program Grant Number 13631.     

Factor analysis for clustered observations

Classical factor analysis assumes a random sample of vectors of observations. For clustered vectors of observations, such as data for students from colleges, or individuals within households, it may be necessary to consider different within-group and between-group factor structures. Such a two-level model for factor analysis is defined, and formulas for a scoring algorithm for estimation with this model are derived. A simple noniterative method based on a decomposition of the total sums of squares and crossproducts is discussed. This method provides a suitable starting solution for the iterative algorithm, but it is also a very good approximation to the maximum likelihood solution. Extensions for higher levels of nesting are indicated. With judicious application of quasi-Newton methods, the amount of computation involved in the scoring algorithm is moderate even for complex problems; in particular, no inversion of matrices with large dimensions is involved. The methods are illustrated on two examples.Suggestions and corrections of three anonymous referees and of an Associate Editor are acknowledged. Discussions with Bob Jennrich on computational aspects were very helpful. Most of research leading to this paper was carried out while the first author was a visiting associate professor at the University of California, Los Angeles.     

Exploratory longitudinal factor analysis in multiple populations

For multiple populatios, a longtidinal factor analytic model which is entirely exploratory, that is, no explicit identification constraints, is proposed. Factorial collapse and period/practice effects are allowed. An invariant and/or stationary factor pattern is permitted. This model is formulated stochastically. To implement this model a stagewise EM algorithm is developed. Finally a numerical illustration utilizing Nesselroade and Baltes' data is presented.The authors wish to thank Barbara Mellers and Henry Kaiser for their helpful comments and John Nesselroade for providing us the data for our illustration. This research wwa supported in part by a grant (No. AG03164) from the National Institute on Aging to William Meredith. Details of the derivations and a copy of the PROC MATRIX program are available upon request from the first author.     

Estimation for structural equation models with missing data

A direct method in handling incomplete data in general covariance structural models is investigated. Asymptotic statistical properties of the generalized least squares method are developed. It is shown that this approach has very close relationships with the maximum likelihood approach. Iterative procedures for obtaining the generalized least squares estimates, the maximum likelihood estimates, as well as their standard error estimates are derived. Computer programs for the confirmatory factor analysis model are implemented. A longitudinal type data set is used as an example to illustrate the results.This research was supported in part by Research Grant DAD1070 from the U.S. Public Health Service. The author is indebted to anonymous reviewers for some very valuable suggestions. Computer funding is provided by the Computer Services Centre, The Chinese University of Hong Kong.     

Relationships among several methods of linearly constrained correspondence analysis

This paper shows essential equivalences among several methods of linearly constrained correspondence analysis. They include Fisher's method of additive scoring, Hayashi's second type of quantification method, ter Braak's canonical correspondence analysis, Nishisato's type of quantification method, ter Braak's canonical correspondence analysis, Nishisato's ANOVA of categorical data, correspondence analysis of manipulated contingency tables, Böckenholt and Böckenholt's least squares canonical analysis with linear constraints, and van der Heijden and Meijerink's zero average restrictions. These methods fall into one of two classes of methods corresponding to two alternative ways of imposing linear constraints, the reparametrization method and the null space method. A connection between the two is established through Khatri's lemma.The work reported in this paper has been supported by grant A6394 from the Natural Sciences and Engineering Research Council of Canada to the first author. We wish to thank Carolyn Anderson, Ulf Böckenholt, Henk Kiers, Shizuhiko Nishisato, Jim Ramsay, Tadashi Shibayama, Cajo ter Braak, and Peter van der Heijden for their helpful comments on earlier drafts of this paper.     

Closing the gap between behavior and models in route choice: The role of spatiotemporal constraints and latent traits in choice set formation

A considerable gap exists between the behavioral paradigm of choice set formation in route choice and its representation in route choice modeling. While travelers form their viable choice set by retaining routes that satisfy spatiotemporal constraints, existing route generation techniques do not account for individual-related spatiotemporal constraints. This paper reduces the gap by proposing a route choice model incorporating spatiotemporal constraints and latent traits. The proposed approach combines stochastic route generation with a latent variable semi-compensatory model representing constraint-based choice set formation followed by compensatory choice. The model is applied to data focusing on habitual commuting route choice behavior in morning peak hours. Results show (i) the possibility of inferring spatiotemporal constraints from considered routes, (ii) the importance of incorporating spatiotemporal constraints and latent traits in route choice models, and (iii) the linkage between spatiotemporal constraints and time saving, spatial and mnemonic abilities.     

A bayesian approach to confirmatory factor analysis

Confirmatory factor analysis is considered from a Bayesian viewpoint, in which prior information on parameter is incorporated in the analysis. An iterative algorithm is developed to obtain the Bayes estimates. A numerical example based on longitudinal data is presented. A simulation study is designed to compare the Bayesian approach with the maximum likelihood method.Computer facilities were provided by the Computer Services Center, The Chinese University of Hong Kong.     

The Effect of Humor on Memory: Constrained by the Pun

In a series of experiments, we investigated the effect of pun humor on memory. In all experiments, the participants were exposed to knock-knock jokes in either the original form retaining the pun or in a modified form that removed the pun. In Experiment 1, the authors found that pun humor improved both recall and recognition memory following incidental encoding. In Experiment 2, they found evidence that rehearsal is not the cause of the humor effect on memory. In Experiments 3 and 4, the authors found that the constraints imposed by puns and incongruity may account for the humor effects observed. Puns constrain and limit the information that can fit in the final line of a joke and thus make recall easier.     

Generating interior search directions for multiobjective linear programming

A new multiobjective linear programming (MOLP) algorithm is presented. The algorithm uses a variant of Karmarkar's interior-point algorithm known as the affine-scaling primal algorithm. Using this single-objective algorithm, interior search directions are generated and used to provide an approximation to the gradient of the (implicitly known) utility function. The approximation is guided by assessing locally relevant preference information for the various interior directions through interaction with a decision maker (DM). The resulting algorithm is an interactive approach that makes its progress towards the solution through the interior of the constraints polytope.     

Individual differences in patterns of developmental opportunity and constraint during COVID-19: Implications for longitudinal well-being

Individual differences in sociodemographic characteristics and trait-like perceptions of opportunities and constraints may shape how people respond to adversities such as the COVID-19 pandemic. However, little is known about how these factors combine to form multifaceted profiles of developmental opportunity and constraint or the implications of such profiles for longitudinal well-being following major life stressors. Using 1-year data from a national sample of U.S. adults (n = 293), we identified profiles based on relevant sociodemographic characteristics (age, socioeconomic status, chronic conditions, functional status) and trait-like perceptions of opportunity and constraints (perceived mastery, perceived constraints). Results of latent profile analyses showed that three common profiles emerged at pandemic onset (veridical opportunity, perceived constraints, perceived opportunity). Subsequent latent growth models showed that, despite reporting more sociodemographic constraints, the perceived opportunity profile exhibited better 1-year emotional well-being (positive and negative affect) during the pandemic than the perceived constraints profile. Findings advance the literature by identifying multifaceted individual differences in profiles of developmental opportunity and constraint and by showing these profiles have consequences for longitudinal well-being following the pandemic onset.     

A general method for proving decidability of intuitionistic modal logics

We generalise the result of [H. Ganzinger, C. Meyer, M. Veanes, The two-variable guarded fragment with transitive relations, in: Proc. 14th IEEE Symposium on Logic in Computer Science, IEEE Computer Society Press, 1999, pp. 24–34] on decidability of the two variable monadic guarded fragment of first order logic with constraints on the guard relations expressible in monadic second order logic. In [H. Ganzinger, C. Meyer, M. Veanes, The two-variable guarded fragment with transitive relations, in: Proc. 14th IEEE Symposium on Logic in Computer Science, IEEE Computer Society Press, 1999, pp. 24–34], such constraints apply to one relation at a time. We modify their proof to obtain decidability for constraints involving several relations. Now we can use this result to prove decidability of multi-modal modal logics where conditions on accessibility relations involve more than one relation. Our main application is intuitionistic modal logic, where the intuitionistic and modal accessibility relations usually interact in a non-trivial way.     

Canonical analysis of two convex polyhedral cones and applications

Canonical analysis of two convex polyhedral cones consists in looking for two vectors (one in each cone) whose square cosine is a maximum. This paper presents new results about the properties of the optimal solution to this problem, and also discusses in detail the convergence of an alternating least squares algorithm. The set of scalings of an ordinal variable is a convex polyhedral cone, which thus plays an important role in optimal scaling methods for the analysis of ordinal data. Monotone analysis of variance, and correspondence analysis subject to an ordinal constraint on one of the factors are both canonical analyses of a convex polyhedral cone and a subspace. Optimal multiple regression of a dependent ordinal variable on a set of independent ordinal variables is a canonical analysis of two convex polyhedral cones as long as the signs of the regression coefficients are given. We discuss these three situations and illustrate them by examples.