Testing Differential Item Functioning in Small Samples

Abstract

Differential item functioning (DIF) is a pernicious statistical issue that can mask true group differences on a target latent construct. A considerable amount of research has focused on evaluating methods for testing DIF, such as using likelihood ratio tests in item response theory (IRT). Most of this research has focused on the asymptotic properties of DIF testing, in part because many latent variable methods require large samples to obtain stable parameter estimates. Much less research has evaluated these methods in small sample sizes despite the fact that many social and behavioral scientists frequently encounter small samples in practice. In this article, we examine the extent to which model complexity—the number of model parameters estimated simultaneously—affects the recovery of DIF in small samples. We compare three models that vary in complexity: logistic regression with sum scores, the 1-parameter logistic IRT model, and the 2-parameter logistic IRT model. We expected that logistic regression with sum scores and the 1-parameter logistic IRT model would more accurately estimate DIF because these models yielded more stable estimates despite being misspecified. Indeed, a simulation study and empirical example of adolescent substance use show that, even when data are generated from / assumed to be a 2-parameter logistic IRT, using parsimonious models in small samples leads to more powerful tests of DIF while adequately controlling for Type I error. We also provide evidence for minimum sample sizes needed to detect DIF, and we evaluate whether applying corrections for multiple testing is advisable. Finally, we provide recommendations for applied researchers who conduct DIF analyses in small samples.     

Photo-synthesis: Facilitating Goal Setting in Supervision Using Images and Creativity

ABSTRACT

One of the most important aspects of clinical supervision is goal-setting. In the early stages of supervision, supervisees may find it difficult to connect authentically with their supervisors due to the inherent power differential in supervision. Difficulty connecting may be exacerbated when students are attempting to set goals regarding their areas for growth. Based on literature reviewed on the counselor-client relationship, the authors posit that using images may increase vulnerability between supervisors and supervisees. Additionally, the authors introduce relational-cultural theory as a framework for connecting authentically through creative images. The authors present examples of using images in individual and group supervision, specifically employing The Coaching Game made by Points of You.     

What Changes in Therapy? Who Changes?

SUMMARY

A central component of therapeutic change involves facilitating the capacity to move and be moved by the other. Another way of saying this might be that change entails experiencing a greater freedom of relational movement. The question of who and what actually changes in the process of therapy is the focus of the three vignettes that follow. They highlight, among other things, the recognition and acknowledgment of mutuality as an essential force within the relational matrix and the ever-changing landscape that this creates. Each of these examples of a change process bears, as well, a particular stamp of its own, and thus speaks to the unique personality of every therapeutic dyad.     

Mapping relational structure in spatial reasoning

Three experiments investigated whether the similarity of relational structures influences the interpretation of spatial representations. Adults were shown diagrams of hand gestures paired with simple statements and asked to judge the meaning of new gestures. In Experiment 1 the gestures were paired with active declarative statements. In Experiment 2, the gestures were paired with conjunctive and disjunctive relations. Experiment 3 used statements similar to those used in Experiment 1, but eliminated the initial object-to-object mapping provided in Experiments 1 and 2. In all three experiments, most participants chose an interpretation that set up a parallel relational structure between the gesture and its meaning: spatial elements were paired with conceptual elements and spatial relations were paired with conceptual relations. These results are consistent with the hypothesis that similarity of relational structures influences spatial reasoning, and have implications for analogical reasoning, diagrammatic reasoning, and language processing.     

Perceiving versus inferring movements to understand dynamic events: The influence of content complexity

ObjectivesWhat type of visual presentation is best in helping learners to understand the functioning of a dynamic system and under what conditions? This study investigated the effect of content complexity on perceived cognitive load and performance resulting from studying depicted movements of team play either in an explicit manner (animation) or via arrow symbols (static diagram).DesignA 2 (treatment: diagram vs. animation) × 2 (content complexity: low vs. high) between subjects design was adopted in the experiment.MethodsForty-eight university students were randomly assigned to the four study conditions and required to perform a reconstruction test and rate their perceived cognitive load following the study phase.ResultsData analyses revealed that for low-complexity content, participants exposed to the animation treatment learned more efficiently – based on the combination of learning and cognitive load scores – than those exposed to the diagram treatment. On the other hand, for high-complexity content, participants exposed to the diagram treatment learned more efficiently than those exposed to the animation treatment.ConclusionThe findings stress the importance of considering the task complexity factor when designing and presenting instructional materials to learners.     

How Therapy Helps When the Culture Hurts

SUMMARY

The purpose of psychotherapy is movement toward relational healing. However, the practice itself is embedded in a culture where relational disconnection and power-over arrangements are normative. The purpose of this article is to examine the impact of cultural disconnections on the therapy relationship. Because they embody multiple social identities within a power-over paradigm, both client and therapist are “carriers” of cultural disconnections. The article examines the shifting vulnerabilities associated with those identities that may lead to impasse and violation or contribute to possibilities for growth. Scenarios from clinical practice illustrate how conflict becomes a pathway to deeper connection when embraced with such processes as empathic attunement, authentic responsiveness, and mutuality.