Testing Differential Item Functioning in Small Samples

Abstract

Differential item functioning (DIF) is a pernicious statistical issue that can mask true group differences on a target latent construct. A considerable amount of research has focused on evaluating methods for testing DIF, such as using likelihood ratio tests in item response theory (IRT). Most of this research has focused on the asymptotic properties of DIF testing, in part because many latent variable methods require large samples to obtain stable parameter estimates. Much less research has evaluated these methods in small sample sizes despite the fact that many social and behavioral scientists frequently encounter small samples in practice. In this article, we examine the extent to which model complexity—the number of model parameters estimated simultaneously—affects the recovery of DIF in small samples. We compare three models that vary in complexity: logistic regression with sum scores, the 1-parameter logistic IRT model, and the 2-parameter logistic IRT model. We expected that logistic regression with sum scores and the 1-parameter logistic IRT model would more accurately estimate DIF because these models yielded more stable estimates despite being misspecified. Indeed, a simulation study and empirical example of adolescent substance use show that, even when data are generated from / assumed to be a 2-parameter logistic IRT, using parsimonious models in small samples leads to more powerful tests of DIF while adequately controlling for Type I error. We also provide evidence for minimum sample sizes needed to detect DIF, and we evaluate whether applying corrections for multiple testing is advisable. Finally, we provide recommendations for applied researchers who conduct DIF analyses in small samples.     

Analyzing the latent factor structure of self-regulation in preschoolers: New directions regarding a differentiation of cool self-regulation

ABSTRACT

Self-regulation in preschool-aged children is a much-researched topic, but there are still disagreements regarding the dimensionality of the construct. Most studies distinguish between hot and cool self-regulation and consider self-regulation as two-factorial. Hot self-regulation tasks refer to the ability to adapt appropriately to emotional stimuli, while cool self-regulation tasks are emotionally neutral. Our study contributes to the understanding of self-regulation at preschool age and reveals new insights concerning a more differentiated view of cool self-regulation. With a total sample of 434 preschool-aged children, we conducted confirmatory and exploratory factor analyses that demonstrated a three-factorial solution. In accordance with the variables that determined the factor structure, we inferred one hot and two cool self-regulation dimensions. We identified a time component in task scoring as a potential reason for the differentiation in cool self-regulation. The novelty of our findings and study limitations are considered.     

Use of trait information in the attribution of intentions by popular,average and rejected children

This study aimed to examine the extent to which children's sociometric status is related to the use of trait information. Therefore, 99 children (aged 4–6) were asked to make inferences about protagonists' future actions when positive or negative trait information was given. Results showed that rejected children were less affected by the protagonist's trait information than their more popular peers (average and popular) in both conditions. As well as their frequently reported hostile bias, rejected children also showed a positive bias. This suggests a general delay in social reasoning among rejected children, but can also be explained by a difference in their development of social cognition based on their atypical daily interactions. Copyright © 2004 John Wiley & Sons, Ltd.     

A step away from impaired well-being: a latent growth curve analysis of an intervention with activity trackers among employees

ABSTRACT

The present study evaluated the effectiveness of a workplace intervention combining activity trackers (behavioural approach) with an online coach (cognitive approach) in order to increase employees’ number of steps and improve their impaired well-being (i.e., emotional strain and negative affect). To analyse the intervention’s effectiveness, the study applied latent growth curve modelling. Moreover, we tested whether work-related and personal resources (i.e., job control and self-efficacy) moderated the intervention’s effectiveness and whether an increase in number of steps was associated with an improvement in impaired well-being. During the intervention, data were collected at six measurement points from 108 mainly low active employees. The results revealed that employees increased their number of steps until the second intervention week; this increase was not moderated by job control or self-efficacy. Moreover, the intervention was effective in decreasing emotional strain and negative affect over the course of the intervention. Further analyses showed that the increase in number of steps was related to the decrease in negative affect, whereas no such association was found for the increase in number of steps and the decrease in emotional strain. In conclusion, the findings showed that our intervention was effective in improving physical activity and impaired well-being among employees.     

A typological approach of perceived resource fluctuations after job transitions in a representative panel study

ABSTRACT

Job and career transitions are unique experiences that vary within and between persons. One possible reason for the differential effects of transitions is that they can involve resource gains, losses, conservation, or a combination thereof. This study investigates perceived resource fluctuation patterns as possible reasons for differential health outcomes in a representative German panel study (n = 2296). Participants compared six characteristics of their new job with their previous one in three categories (better, same, or worse): (1) job security, (2) working hour regulations, (3) workload, (4) use of professional knowledge, (5) advancement opportunities, and (6) earnings. We conducted multilevel latent class analyses (LCA) with mental and physical health as distal outcomes. Results showed a four-class solution with different probabilities of endorsing that job conditions have fluctuated after the transition. Results also indicated important nuances between the latent classes in terms of mental and physical health outcomes. This study adds to a growing body of knowledge concerning the important role resource fluctuations, and the interplay of various resource dynamics play in the sustenance of mental and physical health. Results also provide implications for career guidance, as well as dealing with organizational newcomers to ensure their well-being, and therefore also their performance.     

Estimating Age-Based Developmental Trajectories Using Latent Change Score Models Based on Measurement Occasion

Abstract

Accelerated longitudinal designs (ALDs) are designs in which participants from different cohorts provide repeated measures covering a fraction of the time range of the study. ALDs allow researchers to study developmental processes spanning long periods within a relatively shorter time framework. The common trajectory is studied by aggregating the information provided by the different cohorts. Latent change score (LCS) models provide a powerful analytical framework to analyze data from ALDs. With developmental data, LCS models can be specified using measurement occasion as the time metric. This provides a number of benefits, but has an important limitation: It makes it not possible to characterize the longitudinal changes as a function of a developmental process such as age or biological maturation. To overcome this limitation, we propose an extension of an occasion-based LCS model that includes age differences at the first measurement occasion. We conducted a Monte Carlo study and compared the results of including different transformations of the age variable. Our results indicate that some of the proposed transformations resulted in accurate expectations for the studied process across all the ages in the study, and excellent model fit. We discuss these results and provide the R code for our analysis.     

Investigating curvilinear relationships of explicit and implicit aggression with workplace outcomes

ABSTRACT

Although previous research has demonstrated the value of integrating explicit and implicit measures of aggression for predicting workplace outcomes, such investigations only examined linear and interactive effects. Here we examine nonlinear (i.e., curvilinear) effects of explicit and implicit aggression, which revealed subtleties in the manifestation of aggression. We found significant curvilinear effects of explicit aggression on peer-reported deviance (property deviance, production deviance, and personal aggression). In each case, deviance was lowest when explicit aggression was low, and increased as explicit aggression increased. However, production and property deviance decreased at elevated levels of explicit aggression. In contrast, for disciplinary actions, explicit and implicit aggression interacted, but there was also a curvilinear effect of implicit aggression. This pattern led to discipline being highest when explicit and implicit aggression were very incongruent, but being quite low when they were congruent. The results show that examining nonlinear effects with respect to explicit and implicit aggression can reveal unexpected relationships when predicting peer-reported criteria or organizational personnel data. Propositions for future research on the relationship of personality to workplace outcomes are presented deriving from the trait activation model, and implications for workplace interventions are discussed.     

Distraction and reappraisal efficiency on immediate negative emotional responses: role of trait anxiety

Background and Objectives: Emotion regulation involves attempts to influence emotion unfolding and may target experiential, expressive and physiological responses. Several strategies can be used, such as reappraisal (re-evaluating the emotional situation to reduce its emotional meaning) or distraction (turning the attention toward non-emotional aspects of the situation). Previous research on these regulation strategies produced contrasting results regarding their efficiency and we hypothesize that this could be due to individual differences such as trait anxiety.

Design and Methods: Participants (N?=?77) were confronted with negative pictures and we examined the differential emotional reactivity according to trait anxiety, followed by a comparison of the efficiency of reappraisal and distraction in reducing emotional responses.

Results: Results show that trait anxiety has no impact on reactivity at the experiential and expressive levels, but has an impact at the physiological level, where high anxiety individuals show increased cardiac orienting effect, as well as higher skin conductance and respiratory rate. Regarding regulation, reappraisal and distraction successfully reduce emotional experience and expressivity, but not physiological arousal.

Conclusions: Such contrasting results involve that high trait anxiety individuals might benefit from the use of other kinds of strategies than reappraisal and distraction, some that may successfully target physiological responses.     

Modeling duration of overtaking in two lane highways

The paper attempts to reveal which factors may influence the duration of overtaking in two lane highways. Questions such as what is the duration of young male and female drivers’ overtaking activities and, given that a driver conducts an overtaking maneuver, how long will it take, are addressed using classical survival analysis. Data are collected using a driving simulator. Different models are developed for describing the total overtaking duration, as well as the duration of the acceleration and back-to-lane phases. Results show that the duration of each of the phases of overtaking considered, as well as the total overtaking duration may be best described by a Log–logistic distribution. Analyses point out that, apart from the acceleration phase, the gender is a critical factor to the duration modeling. Other influential factors are the speed difference from the lead vehicle, the speed of opposing traffic, the spacing from the lead and opposing traffic, as well as whether the driver is engaged in multiple overtakes. Finally, the modeling implications to driving assistance systems are discussed.