Multilevel Reliability Measures of Latent Scores Within an Item Response Theory Framework

Abstract

This paper evaluated multilevel reliability measures in two-level nested designs (e.g., students nested within teachers) within an item response theory framework. A simulation study was implemented to investigate the behavior of the multilevel reliability measures and the uncertainty associated with the measures in various multilevel designs regarding the number of clusters, cluster sizes, and intraclass correlations (ICCs), and in different test lengths, for two parameterizations of multilevel item response models with separate item discriminations or the same item discrimination over levels. Marginal maximum likelihood estimation (MMLE)-multiple imputation and Bayesian analysis were employed to evaluate the accuracy of the multilevel reliability measures and the empirical coverage rates of Monte Carlo (MC) confidence or credible intervals. Considering the accuracy of the multilevel reliability measures and the empirical coverage rate of the intervals, the results lead us to generally recommend MMLE-multiple imputation. In the model with separate item discriminations over levels, marginally acceptable accuracy of the multilevel reliability measures and empirical coverage rate of the MC confidence intervals were found in a limited condition, 200 clusters, 30 cluster size, .2 ICC, and 40 items, in MMLE-multiple imputation. In the model with the same item discrimination over levels, the accuracy of the multilevel reliability measures and the empirical coverage rate of the MC confidence intervals were acceptable in all multilevel designs we considered with 40 items under MMLE-multiple imputation. We discuss these findings and provide guidelines for reporting multilevel reliability measures.     

An IRT analysis of motive questionnaires: The Unified Motive Scales

Multiple inventories claiming to assess the same explicit motive (achievement, power, or affiliation) show only mediocre convergent validity. In three studies (N = 1685) the structure, nomological net, and content coverage of multiple existing motive scales was investigated with exploratory factor analyses. The analyses revealed four approach factors (achievement, power, affiliation, and intimacy) and a general avoidance factor with a facet structure. New scales (the Unified Motive Scales; UMS) were developed using IRT, reflecting these underlying dimensions. In comparison to existing questionnaires, the UMS have the highest measurement precision and provide short (six-item) and ultra-short (three-item) scales. In a fourth study (N = 96), the UMS demonstrated incremental validity over existing motive scales with respect to several outcome criteria.     

Latent profile analysis of working memory capacity and thinking styles in adults and adolescents

This study of adults (n = 269) and adolescents (n = 308) applied latent profile analysis to preference for rational and/or experiential cognition, coupled with working memory capacity (WMC). A 4-profile solution comprising rationally dominant, experientially dominant, dual preference, and disengaged groups was retained for both adult and adolescent samples. Our solution indicated that high WMC was associated with both preference and ability to engage in rational processing. Profile membership significantly discriminated both adults and adolescents on several reasoning tasks and measures of cognitive biases. Overall our results indicate that cognitive processing styles and WMC can be combined to create a typology that distinguishes between four types of thinkers who significantly differ in their performance on reasoning tasks.     

An Examination of Network Position and Childhood Relational Aggression: Integrating Resource Control and Social Exchange Theories

Applying resource control theory and social exchange theory, we examined the social network conditions under which elementary age children were likely to engage in relational aggression. Data on classroom peer networks and peer‐nominated behaviors were collected on 671 second‐ through fourth‐grade children in 34 urban, low‐income classrooms. Nested regression models with robust cluster standard errors demonstrated that the association between children's number of relationships and their levels of relational aggression was moderated by the number of relationships that their affiliates had. Children with more peer relationships (i.e., higher network centrality) exhibited higher levels of relational aggression, but only when these relationships were with peers who had fewer connections themselves (i.e., poorly connected peers). This finding remained significant even when controlling for common predictors of relational aggression including gender, overt aggression, prosocial behavior, victimization, social preference, and perceived popularity. Results are discussed in terms of their implications for advancing the literature on childhood relational aggression and their practical applications for identifying children at risk for these behaviors.     

Keeping People In or Out?

Abstract : In this article I argue that the Lutheran doctrine of two kingdoms provides a good basis for the legitimate participation of all citizens in political discourse and decisions, no matter what faith they hold. The argument is backed by selected insights from philosopher Jürgen Habermas and sociologists Linda Woodhead and Paul Heelas. Against this backdrop, I suggest some challenges for a theology that addresses issues related to religion and society.     

Intentions to participate in counselling among front-line,at-risk Irish government employees: an application of the theory of planned behaviour

ABSTRACT

The study set out to examine intentions to engage in counselling among at-risk Irish government employees and the differential utility of two alternative theory of planned behaviour (TPB) models of behaviour to explain intentions to participate in counselling. Individuals (N=259) employed in a front-line, at-risk occupation for the Irish government completed a TPB-based questionnaire. Quantitative analyses revealed that participants held positive to neutral intentions to participate in counselling, irrespective of gender. The original TPB model explained 49.9% of variance in intentions whereas an alternative TPB model, splitting the perceived behavioural control (PBC) construct between its internal and external control components, explained a further 8.9% of variance. Furthermore, self-efficacy was found to be the strongest predictor of intentions. This study therefore supports the use of the TPB in understanding counselling-seeking behaviour.     

Perspective-taking and depth of theory-of-mind reasoning in sequential-move games

Theory-of-mind (ToM) involves modeling an individual's mental states to plan one's action and to anticipate others' actions through recursive reasoning that may be myopic (with limited recursion) or predictive (with full recursion). ToM recursion was examined using a series of two-player, sequential-move matrix games with a maximum of three steps. Participants were assigned the role of Player I, controlling the initial and the last step, or of Player II, controlling the second step. Appropriate for the assigned role, participants either anticipated or planned Player II's strategy at the second step, and then determined Player I's optimal strategy at the first step. Participants more readily used predictive reasoning as Player II (i.e., planning one's own move) than as Player I (i.e., anticipating an opponent's move), although they did not differ when translating reasoning outcome about the second step to optimal action in the first step. Perspective-taking influenced likelihood of predictive reasoning, but it did not affect the rate at which participants acquired it during the experimental block. We conclude that the depth of ToM recursion (related to perspective-taking mechanisms) and rational application of belief-desire to action (instrumental rationality) constitute separate cognitive processes in ToM reasoning.     

The development of automatic and controlled inhibitory retrieval processes in true and false recall

In three experiments, we investigated the role of automatic and controlled inhibitory retrieval processes in true and false memory development in children and adults. Experiment 1 incorporated a directed forgetting task to examine controlled retrieval inhibition. Experiments 2 and 3 used a part-set cue and retrieval practice task to examine automatic retrieval inhibition. In the first experiment, the forget cue had no effect on false recall for adults but reduced false recall for children. In Experiments 2 and 3, both tasks caused retrieval impairments for true and false recall, and this occurred for all age groups. Implicit inhibition, which occurs outside of our conscious control, appears early in childhood. However, because young children do not process false memories as automatically as adults, explicit inhibition can reduce false memory output.     

Art as Orientation

What is it that we lack in everyday life that causes us to value art so highly? This article argues that (almost) all values are to be understood in terms of a needs‐satisfaction system, and hence that the value of art can be understood only with reference to the state of the appreciator prior to engagement with the artwork. Aesthetic appreciation can be analysed as a process, which can be described in empirically based psychological terms, leading to a functional view of aesthetic experience as potentially enhancing the individual's orientation at sensory, cognitive, and interpersonal levels; our motivation is a special form of pleasure. Various objections are then considered, with particular attention to the role of pleasure in connection with tragedy; the relation of art to related cultural pursuits is also briefly indicated.     

The psychological neuroscience of depression: implications for understanding effects of deep brain stimulation

In this article it is suggested that current psychological theories of depression presuppose that this condition will develop as a result of a vicious circle involving negatively biased communication between systems of emotional stress-/alarm-signaling, executive functions and mood regulation. These systems may from a neuroanatomical point of view be located in the limbic system, the orbitofrontal and lateral prefrontal cortex and the hypothalamus-pituitary-adrenal (HPA-) axis respectively. The theoretical and practical implications of this model for the understanding of pharmacological treatments of depression are briefly discussed and this theory is related to the catecholamine hypothesis of depression. The model is furthermore discussed in relation to deep brain stimulation (DBS) of treatment resistant major depression. Similarities and differences between this perspective and the one advocated by the "homeostatic theory" of depression are discussed. It is concluded that a topographical psychological theory may offer a useful heuristic in thinking about depression and that it offers several testable predictions about treatments of the disorder.