Controlling Correlational Bias via Confirmatory Factor Analysis of MTMM Data

Confirmatory factor analysis of multitrait-multimethod (MTMM) data has proven to be a useful tool for assessing convergent and discriminant validity. However, researchers have not made full use of the results of MTMM analyses in examining the relationship between MTMM factors and variables outside the MTMM. Often, researchers simply average the various measures of each trait. Alternatively, they estimate LISREL MTMM models, but estimate only relationships between MTMM traits and the outside variables. In the present article, we show that these two approaches to analyzing data outside the MTMM produce equally highly biased parameter estimates when the actual correlations between MTMM method factors and the outside variables are substantial. An algebraic explanation and a simulated data illustration are given for the bias due to misspecification. Also, the problem is illustrated with a brief empirical example. Implications for applied research are discussed.     

Exploring the relations between traits and emotions

The relationships among traits and among emotions have been well researched and are well known, but few experimental probes of the relationships between traits and emotions have been undertaken. The present investigation sought to describe the relations that exist between traits and emotions. The Comrey Personality Scale (a trait inventory) and the Profile of Mood States (an emotion inventory) were administered to 140 students and the subscales were correlated using canonical procedures. The resulting canonical correlations may be specified in bipolar summary form as follows: Neurotic-Stable was found to be correlated with Depressed-Hopeful; Introverted-Extraverted was found to be correlated with Quiescent-Vigorous; and Empathic-Egocentrical was found to be correlated with Content-Confused. Furthermore, the predictive strength between the trait set and the emotion set was reasonably good, although substantially better when traits rather than emotions served as the predictors.     

Modeling Common Traits and Method Effects in Multitrait-Multimethod Analysis

Method effects often occur when constructs are measured by different methods. In traditional multitrait-multimethod (MTMM) models method effects are regarded as residuals, which implies a mean method effect of zero and no correlation between trait and method effects. Furthermore, in some recent MTMM models, traits are modeled to be specific to a certain method. However, often we are not interested in a method-specific trait but in a trait that is common to all methods. Here we present the Method Effect model with common trait factors, which allows modeling “common” trait factors and method factors that represent method “effects” rather than residuals. The common trait factors are defined as the mean of the true-score variables of all variables measuring the same trait and the method factors are defined as differences between true-score variables and means of true-score variables. Because the model allows estimating mean method effects, correlations between method factors, and correlations between trait and method factors, new research questions may be investigated. The application of the model is demonstrated by 2 examples studying the effect of negative, as compared with positive, item wording for the measurement of mood states.     

On the prediction of academic performance with personality traits: A replication study

We describe a large sample study (N = 652) that had two purposes. The first was to replicate a previous study (Paunonen & Ashton, 2001a, Journal of Research in Personality) in which we showed that lower-level personality trait measures related to achievement motivation predict the academic success of university students. The second purpose was to test our earlier conclusion that such lower-level traits are frequently superior to broader personality factors for accuracy in prediction. Both purposes were met in this study, providing more evidence for our general claim that aggregating narrow personality trait measures into broad factor measures can sometimes have negative consequences regarding the prediction and understanding of important outcomes, including academic performance.     

Analysing multitrait–multimethod data with structural equation models for ordinal variables applying the WLSMV estimator: What sample size is needed for valid results?

Convergent and discriminant validity of psychological constructs can best be examined in the framework of multitrait–multimethod (MTMM) analysis. To gain information at the level of single items, MTMM models for categorical variables have to be applied. The CTC(M?1) model is presented as an example of an MTMM model for ordinal variables. Based on an empirical application of the CTC(M?1) model, a complex simulation study was conducted to examine the sample size requirements of the robust weighted least squares mean‐ and variance‐adjusted χ² test of model fit (WLSMV estimator) implemented in Mplus. In particular, the simulation study analysed the χ² approximation, the parameter estimation bias, the standard error bias, and the reliability of the WLSMV estimator depending on the varying number of items per trait–method unit (ranging from 2 to 8) and varying sample sizes (250, 500, 750, and 1000 observations). The results showed that the WLSMV estimator provided a good – albeit slightly liberal – χ² approximation and stable and reliable parameter estimates for models of reasonable complexity (2–4 items) and small sample sizes (at least 250 observations). When more complex models with 5 or more items were analysed, larger sample sizes of at least 500 observations were needed. The most complex model with 9 trait–method units and 8 items (72 observed variables) requires sample sizes of at least 1000 observations.     

Perfectionistic Automatic Thoughts and Psychological Distress in Adolescents: An Analysis of the Perfectionism Cognitions Inventory

The present paper examines a measure of perfectionistic automatic thoughts in terms of its psychometric properties and association with depression in adolescents. The Perfectionism Cognitions Inventory (PCI) was administered to two samples of adolescents. The first sample of adolescents (N?=?250) also completed measures of trait perfectionism and depression. The second sample of adolescents (N?=?105) completed these same measures as well as measures of negative automatic thoughts, positive automatic thoughts, self-criticism, and dependency. Psychometric analyses established that the PCI consists of one large factor with a high level of internal consistency. As expected, the PCI in adolescents was correlated significantly with trait measures of perfectionism, self-criticism, dependency, as well as general measures of automatic thoughts. Most importantly, a series of hierarchical regressions established that the PCI accounted for a significant degree of unique variance in depression, over and above the variance attributable to trait personality measures and negative automatic thoughts in general. Overall, the findings suggest perfectionistic automatic thoughts can be assessed in a reliable and valid manner in adolescents and that the experience of frequent, perfectionistic thoughts contributes uniquely to increased levels of psychological distress.     

Two tests of the social cognitive model of well-being in Portuguese college students

A social cognitive model of well-being (Lent & Brown, 2006, 2008) was tested in two studies (one cross-sectional, one longitudinal) with Portuguese college students. Participants in Study 1 (N = 366) completed measures of academic self-efficacy, environmental support, goal progress, academic satisfaction and stress, trait positive affect, and overall life satisfaction at a single assessment. Those in Study 2 (N = 158) took the same measures at each of two time points, 15 weeks apart. A modified version of the model improved the fit to the data in Study 1. Optimal fit in Study 2 was achieved with a bidirectional version that modeled reciprocal relations of positive affect to both self-efficacy and support. The predictors in both studies accounted for substantial portions of the variance in academic domain satisfaction, academic stress, and life satisfaction. Implications for future research on the social cognitive model in educational and work settings are considered.     

Scaling of apparent accentedness by magnitude estimation and sensory modality matching

In each of three experiments, 24 students judged the accentedness present in the speech of eight Spanish-English bilinguals.Ss gave magnitude estimations and also squeezed a hand dynamometer to indicate the amount of accentedness in the reading of an English passage by each of the speakers. There was significant agreement amongSs regarding the speech samples with each scaling method, and interscale agreement was good. Power functions fitted to the data had exponents falling in the range expected from earlier psychophysical studies. Scale values correlated significantly with the frequency of accented pronunciations by the speakers as judged by two independent judges. The use of these scaling methods for future research on linguistic features of accent and on the relation between accent and language attitudes is discussed.     

The Use of Multitrait–Multimethod Data for Detecting Nonlinear Relationships: The Case of Psychological Climate and Job Satisfaction

Previous monomethod research has found mixed support for nonlinear effects between certain job characteristics (e.g., job autonomy, job complexity) and outcome variables (e.g., job performance). We hypothesized that these weak nonlinear findings may be due to the prevalence of monomethod research that can lead to a lack of complete measurement and/or introduce common methods variance, either of which may mask the true shape of relationships. Using hierarchical regression analyses and a multitrait–multimethod (MTMM) research design, we found strikingly different results between monomethod and MTMM data when considering the relationships between three psychological climate variables and job satisfaction. While the monomethod results mirrored earlier inconclusive findings, the MTMM data indicated that nonlinear equations explained significantly more of the relationship between all three climate dimensions and job satisfaction. These results suggest that the use of MTMM data allows one to more effectively test for nonlinear effects. Furthermore, these nonlinear results suggest that the format of employee questionnaires should probably change from asking how much an employee has of certain constructs to asking does the employee want more or less of these constructs.     

Test of a social cognitive model of work satisfaction in teachers

Lent and Brown [Lent, R. W., & Brown, S. D. (2006). Integrating person and situation perspectives on work satisfaction: A social-cognitive view. Journal of Vocational Behavior,69, 236-247] recently proposed an integrative model of work satisfaction linked to social cognitive career theory. The model posits that work satisfaction is predicted by five classes of variables: work conditions, goal progress, self-efficacy, goal and efficacy relevant supports, and personality traits (e.g., positive affect). We tested this model in a sample of 366 teachers, finding good overall model-data fit. Of the five predictor classes, work conditions, self-efficacy, and positive affect were each found to explain unique predictive variance. This suggests that teachers who are most satisfied with their jobs see their work environment as supportive, are confident in their abilities to complete work-related tasks and goals, and report high levels of trait positive affect. Findings also offered support for the contention that measures of subjective person-environment fit may not be empirically distinct from work satisfaction. Implications of these findings for future research and applied efforts are considered.     

Narcissism and career success: Occupational self-efficacy and career engagement as mediators

Narcissism is a personality trait that has potentially important effects on career development outcomes, yet empirical research on this issue is sparse. The present study explored the relationships between narcissism and two indicators of career success (i.e., salary and career satisfaction) among a group of young professionals (N = 314). We assessed a model proposing that the effect of narcissism on career success is mediated by increased occupational self-efficacy beliefs and career engagement. While correlations between narcissism and the two indicators of career success were minimal, the results showed a significant indirect effect on salary via occupational self-efficacy and indirect effects on career satisfaction via self-efficacy and career engagement. We discuss the results regarding insights into why narcissism may lead to career success.     

Correspondence between personality trait ratings and behavioral events

To investigate the validity of the trait, Energetic, two studies were conducted in which trait ratings of Energetic and objective measures of performance were recorded. The first study used naive raters (N= 55) who observed pairs of subjects perform in a laboratory setting. There was 84% agreement (p < .001) among the raters as to which person was more Energetic, and people rated as more Energetic performed more energetically on the objectively measured tasks (p < .001). The second study used peer ratings of Energetic and objective measures of energetic behavior in a laboratory setting. There was high agreement among the peers in ratings of Energetic, r = .95 (N = 30) and r = .93 (N = 31) for two groups. Nine men from the upper-quartile and nine men from the lower-quartile of the Energetic scores were tested in the laboratory and were given a structured interview regarding their physical activities. The correlation between peer ratings and laboratory performance was .56 and between peer ratings and the interview data, .64. The results support the position that trait ratings can be valid indicators of regularities in behavior. The procedures used in these studies could be a generally useful model for investigations of the validity of trait ratings.     

A multitrait-multimethod model with minimal assumptions

A new model of confirmatory factor analysis (CFA) for multitrait-multimethod (MTMM) data sets is presented. It is shown that this model can be defined by only three assumptions in the framework of classical psychometric test theory (CTT). All other properties of the model, particularly the uncorrelated-ness of the trait with the method factors are logical consequences of the definition of the model. In the model proposed there are as many trait factors as different traits considered, but the number of method factors is one fewer than the number of methods included in an MTMM study. The covariance structure implied by this model is derived, and it is shown that this model is identified even under conditions under which other CFA-MTMM models are not. The model is illustrated by two empirical applications. Furthermore, its advantages and limitations are discussed with respect to previously developed CFA models for MTMM data.     

Separating trait effects from trait-specific method effects in multitrait-multimethod models: a multiple-indicator CT-C(M-1) model

An overview of several models of confirmatory factor analysis for analyzing multitrait-multimethod (MTMM) data and a discussion of their advantages and limitations are provided. A new class of multi-indicator MTMM models combines several strengths and avoids a number of serious shortcomings inherent in previously developed MTMM models. The new models enable researchers to specify and to test trait-specific-method effects. The trait and method concepts composing these models are explained in detail and are contrasted with those of previously developed MTMM models for multiple indicators. The definitions of the models are explained step by step, and a practical empirical application of the models to the measurement of 3 traits x 3 methods is used to demonstrate their advantages and limitations.     

Examining the within-person effect of affect on daily satisfaction

Assessments of global life satisfaction capture beliefs about overall well-being; state satisfaction assessments focus on short-term or “in-the-moment” appraisals of current life circumstances. Prior research has examined how trait measures of life satisfaction and affect are related at between-person and within-person levels of analysis. At the state level, however, a lack of clarity exists about the nature and magnitude of the association between satisfaction and affect. In a diary study involving assessments of both affect and satisfaction at the daily level (N = 350 with 6024 assessments), we found a consequential effect of affect on state satisfaction due to greater within-person variance over time.     

Changing teachers' beliefs regarding autonomy support and structure: The role of experienced psychological need satisfaction in teacher training

ObjectivesGrounded in Self-Determination Theory, this study examined whether physical education (PE) teachers' psychological need satisfaction experienced during continuous professional development (CPD) on need-supportive teaching predicted changes in their effectiveness and feasibility beliefs regarding the proposed teaching approach, as well as their intentions to apply this approach and subsequent changes in their self-reported in-class behaviors.MethodsPrior to the training, a sample of 80 PE teachers (57.5% men, Mage = 42.70 ± 10.15 years) reported on their effectiveness and feasibility beliefs regarding autonomy-supportive and structuring teaching strategies and their in-class application of these strategies. Immediately following the training, these beliefs were assessed again and participants reported on their psychological need satisfaction experienced during the training and their intentions to apply the proposed strategies. Finally, two weeks after the training, participants' self-reported in-class application of the teaching strategies was measured for the second time.ResultsPsychological need satisfaction experienced during the training related to a change in effectiveness and feasibility beliefs regarding autonomy support and structure, and to teachers' intentions to apply the proposed strategies as reported immediately after receiving the training. In addition, teachers' intentions related to a change in their self-reported in-class application of structuring, but not autonomy-supportive, teaching strategies.ConclusionsExperiences of psychological need satisfaction during CPD can help to increase the likelihood that teachers become more convinced about the effectiveness and feasibility of the proposed change and can produce greater intentions toward change, which may relate to actual (albeit) self-reported behavior change.     

Social support,coping, and depressive symptoms in cardiac illness among Hispanic and non-Hispanic White cardiac patients

In Multi-Trait Multi-Method (MTMM) studies of causal attributions for laboratory events, there is little evidence of convergent and discriminant validity for attribution measures. We report the first MTMM study to investigate the validity of two methods of eliciting causal beliefs for an illness, specifically, myocardial infarction. Adult respondents (N?=?107) listed causes of MI, then completed questionnaire rating scales for causal beliefs for MI. Measures were compared using both Campbell and Fiske's approach to MTMM analyses, and a Confirmatory Factor Analysis approach. Neither single item measures causal beliefs, nor scales of causal beliefs derived using exploratory factor analysis provided much evidence of convergent and discriminant validity. Confirmatory factor analysis showed that a model containing only causal belief factors provided a moderately good fit to the data. Adding a questionnaire method factor significantly improved the fit of the model, as well as substantially changing the pattern of factor loadings: loadings of questionnaire items on causal belief factors were markedly reduced. These results highlight major problems with the measurement of causal beliefs, and in particular question the validity of factor analysis of questionnaire measures of causal beliefs. They also suggest that at least some of the MI causal belief factors reported in the literature are artefacts of common questionnaire method variance.     

Self-Efficacy for Affect Regulation as a Predictor of Future Life Satisfaction and Moderator of the Negative Affect—Life Satisfaction Relationship

Life satisfaction is an important index of mental health and also predicts other important outcomes such as longevity and decreased mortality. Negative affect has a unique inverse relationship with life satisfaction across the life span. Therefore, determining psychological factors that uniquely predict future life satisfaction and that reduce the trait negative affect—life satisfaction relationship is theoretically and clinically important. In light of recent evidence from long-term longitudinal studies that self-efficacy for regulation of negative emotions (SERN) predicts higher future life satisfaction, as well as evidence from a cross-sectional study that a subtype of SERN—self-efficacy for regulating anger—buffers the relationship between trait negative affect and life satisfaction, we tested whether SERN and subtypes of SERN predicted higher life satisfaction and buffered the negative affect—life satisfaction relationship longitudinally over short time periods. After controlling for time 1 life satisfaction, higher time 1 self-efficacy for regulating despondency and distress (SEDes) predicted higher future life satisfaction over average time periods of 17 days (N = 127), 32 days (N = 83), and 41 days (N = 65) among college students. However, in post hoc exploratory regressions that included self-efficacy for experience and expression of positive emotions (SEPos), SEDes predicted higher time 2 life satisfaction, but only SEPos uniquely predicted higher time 3 and time 4 life satisfaction.