Testing Differential Item Functioning in Small Samples

Abstract

Differential item functioning (DIF) is a pernicious statistical issue that can mask true group differences on a target latent construct. A considerable amount of research has focused on evaluating methods for testing DIF, such as using likelihood ratio tests in item response theory (IRT). Most of this research has focused on the asymptotic properties of DIF testing, in part because many latent variable methods require large samples to obtain stable parameter estimates. Much less research has evaluated these methods in small sample sizes despite the fact that many social and behavioral scientists frequently encounter small samples in practice. In this article, we examine the extent to which model complexity—the number of model parameters estimated simultaneously—affects the recovery of DIF in small samples. We compare three models that vary in complexity: logistic regression with sum scores, the 1-parameter logistic IRT model, and the 2-parameter logistic IRT model. We expected that logistic regression with sum scores and the 1-parameter logistic IRT model would more accurately estimate DIF because these models yielded more stable estimates despite being misspecified. Indeed, a simulation study and empirical example of adolescent substance use show that, even when data are generated from / assumed to be a 2-parameter logistic IRT, using parsimonious models in small samples leads to more powerful tests of DIF while adequately controlling for Type I error. We also provide evidence for minimum sample sizes needed to detect DIF, and we evaluate whether applying corrections for multiple testing is advisable. Finally, we provide recommendations for applied researchers who conduct DIF analyses in small samples.     

Scale horizons in analytic hierarchies

In the light of the many criticisms to which the analytic hierarchy process (AHP) of Saaty has been subjected, the authors have produced a refined version, the modified analytic hierarchy process. This paper develops further the ideas behind the modified process and illustrates with examples some of the difficulties in the AHP and their resolution.     

Warriors,authors and baseball coaches: the meaning of metaphor in theories of family therapy

This article examines the metaphors family therapists use in their theories to reveal aspects of the theories which are not explicitly stated, using Whitaker's symbolic experiential therapy, Minuchin's structural therapy and White's narrative therapy as examples. Such examination, drawing on social constructivist understandings of metaphor and meaning making, reveals that Minuchin's metaphors of family as organism and therapist as artist and warrior emphasize the family as relatively holistic and the therapist as relatively interventionist. In contrast, Whitaker's metaphor of family as ecological system or team and therapist as coach emphasizes the interdependence and context sensitivity of the family and relative powerlessness of the therapist to impose change. Finally, White, reflecting his explicitly post‐structural commitment, uses the metaphor of therapy as a journey undertaken with a map and as therapy as an act of re‐narrating a story.     

"易卦"的序结构数学模型在对策理论中的应用

通过分析人类的基本行为元素,并借助于"卦象"序结构数学模型的建立,从而初步讨论了一般谋略方法的产生和起源,以及谋略对策的系统核心元素结构;最后讨论了它们在应用中的初步分类.     

Psychophysics and the anisotropy of time

In psychophysics, experimental control over the presented stimuli is an important prerequisite. Due to the anisotropy of time, this prerequisite is not given in psychophysical experiments on time perception. Many important factors (e.g., the direction of perceived time flow) cannot be manipulated in timing experiments. The anisotropy of time is a peculiarity, which distinguishes the time dimension from other perceptual qualities.Here I summarize the anisotropy-related differences between the perception of time and the perception of other qualities. It is discussed to what extent these differences might affect results and interpretations in psychophysical experiments. In conclusion, I argue for a ‘view from nowhen’ on the psychophysical study of time perception.     

Participatory action research: towards (non-ideal) epistemic justice in a university in South Africa

ABSTRACT

The paper explores the possibilities for promoting epistemic justice in a South African university setting through a participatory action-based photovoice research project in which university researchers worked alongside undergraduate students with no prior experience of doing research. The student voices are employed to understand how learning as capability development and agency expansion can advance epistemic justice in a university setting of hierarchical relationships that make participatory action research challenging. The paper considers how, in this project, spaces of epistemic democracy intersected with the expansion of multidimensional functionings, resulting in more epistemic justice for the student-researchers. The paper considers the possibilities for change through a participatory project toward promoting epistemic justice at the individual level. It also explores some criticisms of the limits of such individual development in the face of structural challenges.     

Testing the factor structure of the Forgiveness Scale among Italian adolescent students

ABSTRACT

The aim of the present study was to analysis the factor structure of the Italian version of the Forgiveness Scale (FS) among 663 adolescent students in Italy. The FS involves of 15 items measuring forgiveness toward the wrongdoer. This scale comprises two subscales, Absence of Negative (AN) and Presence of Positive (PP). These subscales evaluate the absence of negative and the existence of positive emotions, thoughts, and behavioural inclinations. Results suggests that only 11 of the original items are valid in the Italian version of the FS. In contrast to the original study, the two dimensions of forgiveness (AN and PP) were independent of each other in the current analysis. Evidence for convergent validity was also found. The results provide some initial support for the bi-dimensionality and internal consistency of the Italian FS and offer a starting point for future research to be pursued in forgiveness.     

‘Grandpa,Do You Like Roller Coasters?’: Identifying Age‐Appropriate Personality Indicators

Personality development research heavily relies on the comparison of scale means across age. This approach implicitly assumes that the scales are strictly measurement invariant across age. We questioned this assumption by examining whether appropriate personality indicators change over the lifespan. Moreover, we identified which types of items (e.g. dispositions, behaviours, and interests) are particularly prone to age effects. We reanalyzed the German Revised NEO Personality Inventory normative sample (N = 11,724) and applied a genetic algorithm to select short scales that yield acceptable model fit and reliability across locally weighted samples ranging from 16 to 66 years of age. We then examined how the item selection changes across age points and item types. Emotion‐type items seemed to be interchangeable and generally applicable to people of all ages. Specific interests, attitudes, and social effect items—most prevalent within the domains of Extraversion, Agreeableness, and Openness—seemed to be more prone to measurement variations over age. A large proportion of items were systematically discarded by the item‐selection procedure, indicating that, independent of age, many items are problematic measures of the underlying traits. The implications for personality assessment and personality development research are discussed. © 2019 European Association of Personality Psychology