Power and sensitivity of alternative fit indices in tests of measurement invariance

Confirmatory factor analytic tests of measurement invariance (MI) based on the chi-square statistic are known to be highly sensitive to sample size. For this reason, G. W. Cheung and R. B. Rensvold (2002) recommended using alternative fit indices (AFIs) in MI investigations. In this article, the authors investigated the performance of AFIs with simulated data known to not be invariant. The results indicate that AFIs are much less sensitive to sample size and are more sensitive to a lack of invariance than chi-square-based tests of MI. The authors suggest reporting differences in comparative fit index (CFI) and R. P. McDonald's (1989) noncentrality index (NCI) to evaluate whether MI exists. Although a general value of change in CFI (.002) seemed to perform well in the analyses, condition specific change in McDonald's NCI values exhibited better performance than a single change in McDonald's NCI value. Tables of these values are provided as are recommendations for best practices in MI testing.     

Perceptual learning and perceptual search are altered in male university students with higher Autism Quotient scores

Two experiments investigated the validity of the Autism Quotient (AQ) scale for measuring traits associated with Autism Spectrum Disorders (ASD) in a population of male university students. Both studies found evidence that individuals who scored higher on the AQ questionnaire performed in a similar way to individuals with ASD on tasks with a perceptual component. Experiment 1 demonstrated a difference in the degree to which higher scoring AQ individuals performed on a perceptual learning task; with higher AQ scorers showing no advantage for familiar over novel items less. In Experiment 2, higher scorers showed fewer errors when judging the elements, as opposed to the global configuration, of a Navon task letter. Both of these patterns of results had previously been noted in individuals with ASD. These results suggest that the AQ may have some validity in identifying individuals in the nonclinical population with similar performance profiles to those with ASD.     

Time–Frequency Cross Mutual Information Analysis of the Brain Functional Networks Underlying Multiclass Motor Imagery

To study the physiologic mechanism of the brain during different motor imagery (MI) tasks, the authors employed a method of brain-network modeling based on time–frequency cross mutual information obtained from 4-class (left hand, right hand, feet, and tongue) MI tasks recorded as brain–computer interface (BCI) electroencephalography data. The authors explored the brain network revealed by these MI tasks using statistical analysis and the analysis of topologic characteristics, and observed significant differences in the reaction level, reaction time, and activated target during 4-class MI tasks. There was a great difference in the reaction level between the execution and resting states during different tasks: the reaction level of the left-hand MI task was the greatest, followed by that of the right-hand, feet, and tongue MI tasks. The reaction time required to perform the tasks also differed: during the left-hand and right-hand MI tasks, the brain networks of subjects reacted promptly and strongly, but there was a delay during the feet and tongue MI task. Statistical analysis and the analysis of network topology revealed the target regions of the brain network during different MI processes. In conclusion, our findings suggest a new way to explain the neural mechanism behind MI.     

The Factor Structure of Effortful Control and Measurement Invariance Across Ethnicity and Sex in a High-Risk Sample

Measurement invariance of a one-factor model of effortful control (EC) was tested for 853 low-income preschoolers (M age = 4.48 years). Using a teacher-report questionnaire and seven behavioral measures, configural invariance (same factor structure across groups), metric invariance (same pattern of factor loadings across groups), and partial scalar invariance (mostly the same intercepts across groups) were established across ethnicity (European Americans, African Americans and Hispanics) and across sex. These results suggest that the latent construct of EC behaved in a similar way across ethnic groups and sex, and that comparisons of mean levels of EC are valid across sex and probably valid across ethnicity, especially when larger numbers of tasks are used. The findings also support the use of diverse behavioral measures as indicators of a single latent EC construct.     

Cognitive effects of life stress and learned helplessness

Abstract Stressful life events and learned helplessness attributional styles have been shown to impact a variety of personal outcomes. This study examined how these factors influence two classes of cognitive behaviors: the occurrence of intrusive thoughts and performance in memory and verbal-spatial reasoning tasks. Negative life change and attributions for negative events predicted different types of cognitive responses. Individuals reporting higher levels of life stress were more likely to experience distracting thoughts that were unrelated to the current task, whereas individuals with learned helplessness attributional styles tended to have more worrisome thoughts about their task performance. In general, individuals reporting high levels of negative life stress tended to perform more poorly in tasks, whereas individuals with learned helplessness attributional styles tended to perform better than those who did not share this explanatory style. These results suggest that life stress and attributional style have important influecnes on cognitive processes, and that a learned helplessness attributional style can have beneficial effects on behavior in some situations.     

Measurement Invariance of the Wong and Law Emotional Intelligence Scale Scores: Does the Measurement Structure Hold across Far Eastern and European Countries?

In recent years, emotional intelligence and emotional intelligence measures have been used in a plethora of countries and cultures. This is also the case for the Wong and Law Emotional Intelligence Scale (WLEIS), highlighting the importance of examining whether the WLEIS is invariant across regions other than the Far Eastern region (China) where it was originally developed. This study investigated the measurement invariance (MI) of the WLEIS scores across two countries, namely Singapore (N= 505) and Belgium (N= 339). Apart from items measuring the factor “use of emotion”, the measurement structure underlying the WLEIS ratings was generally invariant across both countries as there was no departure from MI in terms of factor form and factor loadings. The scalar invariance model (imposing an identical threshold structure) was partially supported. Factor intercorrelations (not involving the factor “use of emotion”) were also identical across countries. These results show promise for the invariance of the WLEIS scores across different countries, yet warn of the non‐invariance of the dimension “use of emotion”. Reducing the motivation‐oriented nature of these items is in order to come to an exact model fit in cross‐cultural comparisons.     

Level of Education and Category Fluency Task among Spanish Speaking Elders: Number of Words,Clustering, and Switching Strategies

ABSTRACT

It has been well documented that education influences the individual's performance on category fluency tasks but it is still unclear how this effect may differ across the different types of category tasks (i.e., animals, fruits, vegetables and clothing). This study aims (1) to analyze the effect of the level of education on four different types of category fluency tasks among elder Hispanic Americans and (2) to provide normative information on a population with different education levels that was previously screened for neurological and psychiatric conditions. In addition this study examines the semantic strategies used by these individuals to complete the fluency tasks. The sample included 105 healthy Hispanic individuals (age 55–98; 29 males and 76 females) divided into three education groups (<6, 6–11 and >11 years of education). Results showed that after controlling for age and gender, education has a main effect and is a strong predictor of performance in verbal fluency for the categories animals and clothing with increasing educational attainment being associated with higher category fluency scores and with more switches between categories. These findings suggest that the category fruit is less influenced by level of education than the other three semantic categories and may be a more appropriate test across different educational groups. Results from this study provide a reference for clinicians assessing verbal fluency in Spanish speaking populations.     

Differential constraints on the working memory and reading abilities of individuals with learning difficulties and typically developing children

This study examined the factors that constrain the working memory span performance and reading ability of individuals with generalized learning difficulties. In the study, 50 individuals with learning difficulties (LD) and 50 typically developing children (TD) matched for reading age completed two working memory span tasks. Participants also completed independent measures of the processing and storage operations involved in each working memory span task and Raven's Coloured Progressive Matrices. The results showed that despite an equivalent level of working memory span, the relative importance of the constraints on working memory differed between the groups. In addition, working memory span was not closely related to word recognition or sentence comprehension performance in the LD group. These results suggest that the working memory span performance of LD and TD individuals may reflect different working memory limitations and that individuals with generalized learning difficulties may approach cognitive tasks in a qualitatively different way from that of typically developing individuals.     

Individual differences in ego depletion: The role of sociotropy-autonomy

In his cognitive theory of depression, Beck (1987) suggested that highly sociotropic individuals have a strong need for social acceptance whereas highly autonomous individuals have an excessive need for achievement. Research by Baumeister (2000) has suggested that a phenomenon known as ego depletion, the weakening of performance on tasks following active self-control, occurs because it depletes a limited inner resource. The present study examined whether individuals who are highly sociotropic or autonomous would respond differently when faced with tasks requiring self-control. Participants completed the Sociotropy-Autonomy Scale (Clark et al. 1995) and engaged in two active self-control tasks. The results revealed that sociotropy levels were negatively correlated with persistence on tasks that require self-control whereas autonomy was positively correlated to persistence on the same task. In addition, the results suggested that, following a task requiring self-control, highly sociotropic individuals expend less effort, whereas highly autonomous individuals expend more effort on subsequent tasks requiring self-control.     

Measuring stress in clinical and nonclinical subjects using a German adaptation of the Perceived Stress Scale

Background/Objective: Stress is perceived differently across individuals, which might be particularly true for nonclinical and clinical subjects. For this reason, we tested a German adaption of the 10-item Perceived Stress Scale (PSS-10) for model fit and measurement invariance in a big nonclinical and clinical sample. Method: We (1) conducted multiple confirmatory factor analysis (CFA) in 1,248 nonclinical subjects and 575 outpatients, (2) measurement invariance with multigroup CFA, (3) assessed correlations with relevant constructs and (4) calculated internal consistencies for overall stress and the subscales Helplessness and Self-efficacy. Results: In both samples, CFA revealed a robust two-factorial structure with an excellent model fit. Group comparisons revealed strict measurement invariance. Correlations with associated measures support validity. Internal consistencies were good to very good. Conclusions: We show highly satisfactory psychometric properties of the German PSS-10 for nonclinical and clinical individuals. Measurement invariance analyses demonstrated that varying stress levels of people with a different mental health status are due to true interindividual differences.     

The influence of task paradigm on motor imagery ability in children with Developmental Coordination Disorder

Children with Developmental Coordination Disorder (DCD) have difficulty imagining movements such that they conform to the customary temporal constraints of real performance. We examined whether this ability is influenced by the choice of task used to elicit motor imagery (MI). Performance of typically developing (TD) (n = 30) and children with DCD (n = 30) was compared on two tasks: the Visually Guided Pointing Task (VGPT) and the Computerized Virtual Radial Fitts Task (C-VRFT). Since the VGPT places higher demands on executive functions like working memory but requires less spatial planning, we reasoned that the C-VRFT would provide a purer measure of motor imagery (or simulation). Based on our earlier work, we predicted that imagery deficits in DCD would more likely manifest on the C-VRFT. Results showed high correlations between tasks in terms of executed and imagined movement time suggest that both tasks measure MI ability. However, group differences were more pronounced in the imagined condition of the radial Fitts’ task. Taken together, the more spatially complex C-VRFT appears to be a more sensitive measure of motor imagery, better discriminating between DCD and TD. Implications for theory and practice are discussed.     

Structure and Measurement Invariance of Adult Romantic Attachment

Romantic attachment is a popular theory for explaining affect, cognition, and behavior in romantic contexts. This popularity has led to a surge of self-report measures assessing dimensions of attachment. In this study, we considered the ability of 2 common attachment measures, the Adult Attachment Questionnaire (AAQ) and the Experience in Close Relationships–Revised (ECR–R), to replicate the avoidant and anxious attachment factors. We also determined the degree of measurement invariance across, and mean differences between, genders and single and nonsingle individuals. Both the AAQ (N = 650) and the ECR–R (N = 1,271) successfully distinguished avoidant and attachment factors. The AAQ showed evidence for partial strong measurement invariance, whereas the ECR-R showed strict factorial invariance for both gender and relationship status. Gender differences were detected on both measures in a direction consistent with previous research, with males exhibiting higher levels of avoidant attachment (relative to females) and females exhibiting higher levels of anxious attachment (relative to males). Furthermore, when compared to individuals who were currently single, those in romantic relationships exhibited lower levels of avoidant tendencies. This research aligns with the notion that the AAQ and ECR–R reliably assess similar constructs, across genders and single and nonsingle individuals.     

Evaluating information processing in Autism Spectrum Disorder: The case for Fuzzy Trace Theory

Literature on the developmental trajectory of cognition in Autism Spectrum Disorders (ASD) presents several inconsistent findings. In this review, we focus on information processing, and specifically, the use of gist and verbatim representations to guide memory, reasoning, and concept formation. The added perspective of Fuzzy Trace Theory may help to resolve mixed results regarding the frequency, nature, and effectiveness of gist processing in ASD by providing established process models and tasks suitable for use in individuals at all levels of functioning. In order to demonstrate the utility of FTT to the field of ASD research, we present evidence from three complementary theories–Weak Central Coherence (WCC), Executive Function (EF), and Theory of Mind (ToM)–which have dominated the landscape to date. We discuss the potential utility of FTT tasks and process models, proposing testable hypotheses that address ambiguous or conflicting results in the current literature. Applying a theory of typical development to the study of ASD may add value to past and future research.     

Adapted to explore: Reinforcement learning in Autistic Spectrum Conditions

Recent studies have recorded a tendency of individuals with Autism Spectrum Conditions (ASC) to continually change their choices in repeated choice tasks. In the current study we examine if this finding implies that ASC individuals have a cognitive style that facilitates exploration and discovery. Six decision tasks were administered to adolescents with ASC and matched controls. Significant differences in shifting between choice options appeared in the Iowa Gambling task (Bechara, Damasio, Damasio, & Anderson, 1994). A formal cognitive modeling analysis demonstrated that for about half of the ASC participants the adaptation process did not conform to the standard reinforcement learning model. These individuals were only coarsely affected by choice-outcomes, and were more influenced by the exploratory value of choices, being attracted to previously un-explored alternatives. An examination of the five simpler decision tasks where the advantageous option was easier to determine showed no evidence of this pattern, suggesting that the shifting choice pattern is not an uncontrollable tendency independent of task outcomes. These findings suggest that ASC individuals have a unique adaptive learning style, which may be beneficial is some learning environment but maladaptive in others, particularly in social contexts.     

Differences in the use of autobiographical memory across the adult lifespan

Current research distinguishes between self, directive and social function of autobiographical memories (AMs). To date, only few studies have investigated these functions across adulthood. The comparison of different age groups requires that the functions of AM are measured in the same way across groups (measurement invariance, MI). Additionally to the average use of AM, the factor variances and factor covariances among the three functions were examined across adulthood. In the present study, 1290 adults (aged between 17 and 93 years) completed the Thinking about Life Experiences Questionnaire (TALE), which measures the overall use of AMs for self, directive and social purposes. The sample was divided into five age groups and partial strong MI was established using multigroup confirmatory factor analysis. The results showed an increase in the factor associations as well as a decrease in the factor mean levels of all three functions across age groups. Both findings could be adequately described by linear functions of age. The factor variances were on most parts equal across age groups. These results strengthen the assumption that—from a lifespan developmental perspective—the use of AM may be aligned with relatively normative developmental tasks in a given society.     

Invariance in Measurement and Prediction Revisited

Borsboom (Psychometrika, 71:425–440, 2006) noted that recent work on measurement invariance (MI) and predictive invariance (PI) has had little impact on the practice of measurement in psychology. To understand this contention, the definitions of MI and PI are reviewed, followed by results on the consistency between the two forms of invariance in the general case. The special parametric cases of factor analysis (strict factorial invariance) and linear regression analyses (strong regression invariance) are then described, along with findings on the inconsistency between the two forms of invariance in this context. Two numerical examples of inconsistency are reviewed in detail. The impact of violations of MI on accuracy of selection is illustrated. Finally, reasons for the slow dissemination of work on invariance are discussed, and the prospects for altering this situation are weighed. This paper is based on the Presidential Address given at the International Meeting of the Psychometric Society in Tokyo, Japan, on July 11, 2007. This research was supported by National Institute of Mental Health grants 1P30 MH 068685-01A1 and RO1 MH64707-01.     

The Questionnaire Big Six in 26 Nations: Developing Cross‐Culturally Applicable Big Six,Big Five and Big Two Inventories

The Big Five is a useful model of attributes now commonly used in cross‐cultural research, but without the support of strong measurement invariance (MI) evidence. The Big Six has been proposed as a cross‐culturally informed update, and the broader Big Two (Social Self‐Regulation and Dynamism) draws on even more cross‐cultural evidence. However, neither has been rigorously tested for cross‐cultural MI. Here a Big Six inventory (36QB6) and measures of the Big Five and Big Two derived from it were tested and refined for cross‐cultural usability in samples from 26 nations, divided into three subsets. Confirmatory factor analysis of the models in the first subset of nations demonstrated fit as strong in translation as typical personality measures achieve in their nation of origin (although poor per standard benchmarks). Items that performed inconsistently across cultures were removed, and alternates considered in a second subset of nations. Fit and invariance were improved for refined 30‐item QB6, 25‐item Big Five and 14‐item Big Two measures in the third subset of nations. For all models, decrease in comparative fit index between MI levels was larger than .01, indicating lack of support for higher levels. Configural and factorial invariance were relatively stronger, compared to scalar and full. Copyright © 2014 European Association of Personality Psychology     

Reconsidering the Equivalence of Multisource Performance Ratings: Evidence for the Importance and Meaning of Rater Factors

Purpose

The study specified an alternate model to examine the measurement invariance of multisource performance ratings (MSPRs) to systematically investigate the theoretical meaning of common method variance in the form of rater effects. As opposed to testing invariance based on a multigroup design with raters aggregated within sources, this study specified both performance dimension and idiosyncratic rater factors.

Design/Methodology/Approach

Data was obtained from 5,278 managers from a wide range of organizations and hierarchical levels, who were rated on the BENCHMARKS^® MSPR instrument.

Findings

Our results diverged from prior research such that MSPRs were found to lack invariance for raters from different levels. However, same level raters provided equivalent ratings in terms of both the performance dimension loadings and rater factor loadings.

Implications

The results illustrate the importance of modeling rater factors when investigating invariance and suggest that rater factors reflect substantively meaningful variance, not bias.

Originality/Value

The current study applies an alternative model to examine invariance of MSPRs that allowed us to answer three questions that would not be possible with more traditional multigroup designs. First, the model allowed us to examine the impact of paramaterizing idiosyncratic rater factors on inferences of cross-rater invariance. Next, including multiple raters from each organizational level in the MSPR model allowed us to tease apart the degree of invariance in raters from the same source, relative to raters from different sources. Finally, our study allowed for inferences with respect to the invariance of idiosyncratic rater factors.     

Development of the Ability to Insert into a Series

The development of children's ability to insert an item into a series was investigated. Children ranging in age from 3 to 6 years old were given a seriation construction and insertion task, several different types of insertion tasks, and tasks in which they learned to recognize a series. Children's performance on the seriation construction and insertion task was not influenced by their experience with other tasks, and they were able to solve different types of insertion tasks by 5 years of age. The results thus support Piaget's (1965) and Halford's (1982) predictions of stage-related constraints on children's insertion skills, rather than Bullock's (1985) invariance hypothesis, which claims that experience is a more important influence.     

The Childhood Executive Functioning Inventory (CHEXI): Factor structure,measurement invariance,and correlates in US preschoolers

This study tests the factor structure, measurement invariance, and correlates of the Childhood Executive Functioning Inventory (CHEXI) with a large and diverse sample of 3- to 5-year-olds (n = 844). Consistent with previous studies, a two-factor model that distinguishes working memory from inhibition provides the best fit to the observed data. This two-factor model has been shown to demonstrate strong measurement invariance for different subgroups of children (boys vs. girls, high vs. low income). Whereas boys tend to have greater working memory and inhibition difficulties (Cohen’s d = 0.15 and 0.20, respectively), children from low-income households tend to have more working memory problems than their peers from high-income households (Cohen’s d = 0.25). Finally, correlations between CHEXI scores, examiner reports of child behavior, and child performance on a battery of executive function (EF) tasks were investigated. CHEXI scores were found to be more consistently related to examiner reports of child behavior than child performance on EF tasks. Tthe strengths and weaknesses of the CHEXI as a questionnaire measure of EF are discussed, and directions for future research are suggested.