Bayesian generalized structured component analysis

Generalized structured component analysis (GSCA) is a component-based approach to structural equation modelling, which adopts components of observed variables as proxies for latent variables and examines directional relationships among latent and observed variables. GSCA has been extended to deal with a wider range of data types, including discrete, multilevel or intensive longitudinal data, as well as to accommodate a greater variety of complex analyses such as latent moderation analysis, the capturing of cluster-level heterogeneity, and regularized analysis. To date, however, there has been no attempt to generalize the scope of GSCA into the Bayesian framework. In this paper, a novel extension of GSCA, called BGSCA, is proposed that estimates parameters within the Bayesian framework. BGSCA can be more attractive than the original GSCA for various reasons. For example, it can infer the probability distributions of random parameters, account for error variances in the measurement model, provide additional fit measures for model assessment and comparison from the Bayesian perspectives, and incorporate external information on parameters, which may be obtainable from past research, expert opinions, subjective beliefs or knowledge on the parameters. We utilize a Markov chain Monte Carlo method, the Gibbs sampler, to update the posterior distributions for the parameters of BGSCA. We conduct a simulation study to evaluate the performance of BGSCA. We also apply BGSCA to real data to demonstrate its empirical usefulness.     

Generalized Structured Component Analysis with Latent Interactions

Generalized structured component analysis (GSCA) is a component-based approach to structural equation modeling. In practice, researchers may often be interested in examining the interaction effects of latent variables. However, GSCA has been geared only for the specification and testing of the main effects of variables. Thus, an extension of GSCA is proposed to effectively deal with various types of interactions among latent variables. In the proposed method, a latent interaction is defined as a product of interacting latent variables. As a result, this method does not require the construction of additional indicators for latent interactions. Moreover, it can easily accommodate both exogenous and endogenous latent interactions. An alternating least-squares algorithm is developed to minimize a single optimization criterion for parameter estimation. A Monte Carlo simulation study is conducted to investigate the parameter recovery capability of the proposed method. An application is also presented to demonstrate the empirical usefulness of the proposed method.     

Functional Generalized Structured Component Analysis

An extension of Generalized Structured Component Analysis (GSCA), called Functional GSCA, is proposed to analyze functional data that are considered to arise from an underlying smooth curve varying over time or other continua. GSCA has been geared for the analysis of multivariate data. Accordingly, it cannot deal with functional data that often involve different measurement occasions across participants and a large number of measurement occasions that exceed the number of participants. Functional GSCA addresses these issues by integrating GSCA with spline basis function expansions that represent infinite-dimensional curves onto a finite-dimensional space. For parameter estimation, functional GSCA minimizes a penalized least squares criterion by using an alternating penalized least squares estimation algorithm. The usefulness of functional GSCA is illustrated with gait data.     

Dynamic GSCA (Generalized Structured Component Analysis) with Applications to the Analysis of Effective Connectivity in Functional Neuroimaging Data

We propose a new method of structural equation modeling (SEM) for longitudinal and time series data, named Dynamic GSCA (Generalized Structured Component Analysis). The proposed method extends the original GSCA by incorporating a multivariate autoregressive model to account for the dynamic nature of data taken over time. Dynamic GSCA also incorporates direct and modulating effects of input variables on specific latent variables and on connections between latent variables, respectively. An alternating least square (ALS) algorithm is developed for parameter estimation. An improved bootstrap method called a modified moving block bootstrap method is used to assess reliability of parameter estimates, which deals with time dependence between consecutive observations effectively. We analyze synthetic and real data to illustrate the feasibility of the proposed method.     

Out-of-bag Prediction Error: A Cross Validation Index for Generalized Structured Component Analysis

Cross validation is a useful way of comparing predictive generalizability of theoretically plausible a priori models in structural equation modeling (SEM). A number of overall or local cross validation indices have been proposed for existing factor-based and component-based approaches to SEM, including covariance structure analysis and partial least squares path modeling. However, there is no such cross validation index available for generalized structured component analysis (GSCA) which is another component-based approach. We thus propose a cross validation index for GSCA, called Out-of-bag Prediction Error (OPE), which estimates the expected prediction error of a model over replications of so-called in-bag and out-of-bag samples constructed through the implementation of the bootstrap method. The calculation of this index is well-suited to the estimation procedure of GSCA, which uses the bootstrap method to obtain the standard errors or confidence intervals of parameter estimates. We empirically evaluate the performance of the proposed index through the analyses of both simulated and real data.     

Multilevel Dynamic Generalized Structured Component Analysis for Brain Connectivity Analysis in Functional Neuroimaging Data

We extend dynamic generalized structured component analysis (GSCA) to enhance its data-analytic capability in structural equation modeling of multi-subject time series data. Time series data of multiple subjects are typically hierarchically structured, where time points are nested within subjects who are in turn nested within a group. The proposed approach, named multilevel dynamic GSCA, accommodates the nested structure in time series data. Explicitly taking the nested structure into account, the proposed method allows investigating subject-wise variability of the loadings and path coefficients by looking at the variance estimates of the corresponding random effects, as well as fixed loadings between observed and latent variables and fixed path coefficients between latent variables. We demonstrate the effectiveness of the proposed approach by applying the method to the multi-subject functional neuroimaging data for brain connectivity analysis, where time series data-level measurements are nested within subjects.     

Regularized ELM bagging model for Tropical Cyclone Tracks prediction in South China Sea

This paper aims to improve the prediction accuracy of Tropical Cyclone Tracks (TCTs) over the South China Sea (SCS) with 24 h lead time. The model proposed in this paper is a regularized extreme learning machine (ELM) ensemble using bagging. The method which turns the original problem into quadratic programming (QP) problem is proposed in this paper to solve lasso and elastic net problem in ELM. The forecast error of TCTs data set is the distance between real position and forecast position. Compared with the stepwise regression method widely used in TCTs, 8.26 km accuracy improvement is obtained by our model based on the dataset with 70/1680 testing/training records. By contrast, the improvement using this model is 16.49 km based on a smaller dataset with 30/720 testing/training records. Results show that the regularized ELM bagging has a general better generalization capacity on TCTs data set.     

Structural equation modeling with small sample sizes using two-stage ridge least-squares estimation

In covariance structure analysis, two-stage least-squares (2SLS) estimation has been recommended for use over maximum likelihood estimation when model misspecification is suspected. However, 2SLS often fails to provide stable and accurate solutions, particularly for structural equation models with small samples. To address this issue, a regularized extension of 2SLS is proposed that integrates a ridge type of regularization into 2SLS, thereby enabling the method to effectively handle the small-sample-size problem. Results are then reported of a Monte Carlo study conducted to evaluate the performance of the proposed method, as compared to its nonregularized counterpart. Finally, an application is presented that demonstrates the empirical usefulness of the proposed method.     

A comparative evaluation of factor- and component-based structural equation modelling approaches under (in)correct construct representations

Structural equation modelling (SEM) has evolved into two domains, factor-based and component-based, dependent on whether constructs are statistically represented as common factors or components. The two SEM domains are conceptually distinct, each assuming their own population models with either of the statistical construct proxies, and statistical SEM approaches should be used for estimating models whose construct representations correspond to what they assume. However, SEM approaches have often been evaluated and compared only under population factor models, providing misleading conclusions about their relative performance. This is partly because population component models and their relationships have not been clearly formulated. Also, it is of fundamental importance to examine how robust SEM approaches can be to potential misrepresentation of constructs because researchers may often lack clear theories to determine whether a factor or component is more representative of a given construct. Addressing these issues, this study begins by clarifying several population component models and their relationships and then provides a comprehensive evaluation of four SEM approaches – the maximum likelihood approach and factor score regression for factor-based SEM as well as generalized structured component analysis (GSCA) and partial least squares path modelling (PLSPM) for component-based SEM – under various experimental conditions. We confirm that the factor-based SEM approaches should be preferred for estimating factor models, whereas the component-based SEM approaches should be chosen for component models. Importantly, the component-based approaches are generally more robust to construct misrepresentation than the factor-based ones. Of the component-based approaches, GSCA should be chosen over PLSPM, regardless of whether or not constructs are misrepresented.     

Functional Extended Redundancy Analysis

We propose a functional version of extended redundancy analysis that examines directional relationships among several sets of multivariate variables. As in extended redundancy analysis, the proposed method posits that a weighed composite of each set of exogenous variables influences a set of endogenous variables. It further considers endogenous and/or exogenous variables functional, varying over time, space, or other continua. Computationally, the method reduces to minimizing a penalized least-squares criterion through the adoption of a basis function expansion approach to approximating functions. We develop an alternating regularized least-squares algorithm to minimize this criterion. We apply the proposed method to real datasets to illustrate the empirical feasibility of the proposed method.     

Exploratory factor analysis for small samples

Traditionally, two distinct approaches have been employed for exploratory factor analysis: maximum likelihood factor analysis and principal component analysis. A third alternative, called regularized exploratory factor analysis, was introduced recently in the psychometric literature. Small sample size is an important issue that has received considerable discussion in the factor analysis literature. However, little is known about the differential performance of these three approaches to exploratory factor analysis in a small sample size scenario. A simulation study and an empirical example demonstrate that regularized exploratory factor analysis may be recommended over the two traditional approaches, particularly when sample sizes are small (below 50) and the sample covariance matrix is near singular.     

Regularized Multiple-Set Canonical Correlation Analysis

Multiple-set canonical correlation analysis (Generalized CANO or GCANO for short) is an important technique because it subsumes a number of interesting multivariate data analysis techniques as special cases. More recently, it has also been recognized as an important technique for integrating information from multiple sources. In this paper, we present a simple regularization technique for GCANO and demonstrate its usefulness. Regularization is deemed important as a way of supplementing insufficient data by prior knowledge, and/or of incorporating certain desirable properties in the estimates of parameters in the model. Implications of regularized GCANO for multiple correspondence analysis are also discussed. Examples are given to illustrate the use of the proposed technique. The work reported in this paper is supported by Grants 10630 and 290439 from the Natural Sciences and Engineering Research Council of Canada to the first and the second authors, respectively. The authors would like to thank the two editors (old and new), the associate editor, and four anonymous reviewers for their insightful comments on earlier versions of this paper. Matlab programs that carried out the computations reported in the paper are available upon request.     

Two-Way Regularized Fuzzy Clustering of Multiple Correspondence Analysis

Multiple correspondence analysis (MCA) is a useful tool for investigating the interrelationships among dummy-coded categorical variables. MCA has been combined with clustering methods to examine whether there exist heterogeneous subclusters of a population, which exhibit cluster-level heterogeneity. These combined approaches aim to classify either observations only (one-way clustering of MCA) or both observations and variable categories (two-way clustering of MCA). The latter approach is favored because its solutions are easier to interpret by providing explicitly which subgroup of observations is associated with which subset of variable categories. Nonetheless, the two-way approach has been built on hard classification that assumes observations and/or variable categories to belong to only one cluster. To relax this assumption, we propose two-way fuzzy clustering of MCA. Specifically, we combine MCA with fuzzy k-means simultaneously to classify a subgroup of observations and a subset of variable categories into a common cluster, while allowing both observations and variable categories to belong partially to multiple clusters. Importantly, we adopt regularized fuzzy k-means, thereby enabling us to decide the degree of fuzziness in cluster memberships automatically. We evaluate the performance of the proposed approach through the analysis of simulated and real data, in comparison with existing two-way clustering approaches.     

Regularized Generalized Canonical Correlation Analysis

Regularized generalized canonical correlation analysis (RGCCA) is a generalization of regularized canonical correlation analysis to three or more sets of variables. It constitutes a general framework for many multi-block data analysis methods. It combines the power of multi-block data analysis methods (maximization of well identified criteria) and the flexibility of PLS path modeling (the researcher decides which blocks are connected and which are not). Searching for a fixed point of the stationary equations related to RGCCA, a new monotonically convergent algorithm, very similar to the PLS algorithm proposed by Herman Wold, is obtained. Finally, a practical example is discussed.     

Exploring the peak and end rule of past affective episodes within the exercise context

ObjectivesThe global affective evaluation (GAE) of an event influences the decision to repeat that event. Two moments are proposed to predict the GAE: the peak and end affect experienced from the event (Fredrickson, 2000). The purposes of this study were to test this peak and end rule in the context of exercise and examine the relationship between GAE and exercise behaviour.Methods41 inactive women (M = 42.7 years, SD = 10.6 years) completed a graded exercise test to determine ventilatory threshold (VT) and 20 min of treadmill exercise at an intensity either at-VT or 10% above-VT. Feeling Scale (FS) was recorded every 2 min during-exercise and 1, 5, 10 and 15 min post-exercise. GAE was measured 5 min, 15 min, 2 and 7 days post-exercise. Exercise intentions and behaviour were measured 7 days post-exercise. The individual's peak and end FS values were entered together as predictor variables in separate regression analyses with GAE from each time point as the dependent variable.ResultsPeak affect and end affect explained between 39 and 58% of the variance in GAE. Greater variance was predicted 5 and 15 min post-exercise compared with 2 and 7 days post-exercise. The independent contribution of the peak and end variables could not be determined due to multi-collinearity problems. No significant relationships existed between affective memory and intentions or behaviour.ConclusionsThe peak and end rule plays some part in predicting the affective memory of an exercise experience but other variables are likely to play a role.     

Deconstructing trait anxiety: a network perspective

Background and objectives: For decades, the dominant paradigm in trait anxiety research has regarded the construct as signifying the underlying cause of the thoughts, feelings, and behaviors that supposedly reflect its presence. Recently, a network theory of personality has appeared. According to this perspective, trait anxiety is a formative construct emerging from interactions among its constitutive features (e.g., thought, feelings, behaviors); it is not a latent cause of these features.

Design: In this study, we characterized trait anxiety as a network system of interacting elements.

Methods: To do so, we estimated a graphical gaussian model via the computation of a regularized partial correlation network in an unselected sample (N?=?611). We also implemented modularity-based community detection analysis to test whether the features of trait anxiety cohere as a single network system.

Results: We find that trait anxiety can indeed be conceptualized as a single, coherent network system of interacting elements.

Conclusions: This radically new approach to visualizing trait anxiety may offer an especially informative view of the interplay between its constitutive features. As prior research has implicated trait anxiety as a risk factor for the development of anxiety-related psychopathology, our findings also set the scene for novel research directions.     

On optimal smoothing and derivative estimation from noisy displacement data in biomechanics

Estimating derivatives from noisy displacement data is a notoriously ill-posed problem in signal processing and biomechanics. Following the work of Wood and Jennings (1978) and Hatze (1979, 1981), the present paper describes the use of optimally regularized, natural quintic splines for estimating smoothed positions, velocities, and accelerations from equidistantly sampled, noisy position measurements. It appears that the nature of the boundary conditions of the data is of some importance, since various algorithms used hitherto result in artefacts throughout the data if the true derivatives at the record ends differ significantly from zero. Natural quintic splines do not suffer from this disadvantage below the third derivative.The ill-posed character of movement analysis has some interesting implications for movement synthesis and optimization, similar to the indeterminacy of muscular co-contraction from merely external, biomechanical measurements.     

Bayesian multidimensional scaling for the estimation of a Minkowski exponent

The Minkowski property of psychological space has long been of interest to researchers. A common strategy has been calculating the stress in multidimensional scaling for many Minkowski exponent values and choosing the one that results in the lowest stress. However, this strategy has an arbitrariness problem—that is, a loss function. Although a recently proposed Bayesian approach could solve this problem, the method was intended for individual subject data. It is unknown whether this method is directly applicable to averaged or single data, which are common in psychology and behavioral science. Therefore, we first conducted a simulation study to evaluate the applicability of the method to the averaged data problem and found that it failed to recover the true Minkowski exponent. Therefore, a new method is proposed that is a simple extension of the existing Euclidean Bayesian multidimensional scaling to the Minkowski metric. Another simulation study revealed that the proposed method could successfully recover the true Minkowski exponent. BUGS codes used in this study are given in the Appendix.