Multilevel Reliability Measures of Latent Scores Within an Item Response Theory Framework

Abstract

This paper evaluated multilevel reliability measures in two-level nested designs (e.g., students nested within teachers) within an item response theory framework. A simulation study was implemented to investigate the behavior of the multilevel reliability measures and the uncertainty associated with the measures in various multilevel designs regarding the number of clusters, cluster sizes, and intraclass correlations (ICCs), and in different test lengths, for two parameterizations of multilevel item response models with separate item discriminations or the same item discrimination over levels. Marginal maximum likelihood estimation (MMLE)-multiple imputation and Bayesian analysis were employed to evaluate the accuracy of the multilevel reliability measures and the empirical coverage rates of Monte Carlo (MC) confidence or credible intervals. Considering the accuracy of the multilevel reliability measures and the empirical coverage rate of the intervals, the results lead us to generally recommend MMLE-multiple imputation. In the model with separate item discriminations over levels, marginally acceptable accuracy of the multilevel reliability measures and empirical coverage rate of the MC confidence intervals were found in a limited condition, 200 clusters, 30 cluster size, .2 ICC, and 40 items, in MMLE-multiple imputation. In the model with the same item discrimination over levels, the accuracy of the multilevel reliability measures and the empirical coverage rate of the MC confidence intervals were acceptable in all multilevel designs we considered with 40 items under MMLE-multiple imputation. We discuss these findings and provide guidelines for reporting multilevel reliability measures.     

Personality trait inferences of Turkish immigrant and neutral targets: An experimental study

Sandal, G. M. Bye, H. H. & Pallesen, S. (2012). Personality trait inferences of Turkish immigrant and neutral targets: An experimental study. Scandinavian Journal of Psychology 53, 528–533. The study investigated whether personality traits attributed to immigrant targets differ from personality inferences made for a neutral target, and whether trait attributions differ for assimilated and integrated immigrant targets. Participants (n = 340) were randomized to one of three conditions in which they read the same story about a person, but where the person was described as either: (a) an assimilated Turkish immigrant; (b) an integrated Turkish immigrant; or (c) neutral (no nationality or religious practice indicated). Subsequently, they rated the personality of the described person on the NEO‐Five Factor Inventory (observer rating version) and completed the Balanced Inventory of Desirable Responding (Impression Management scale) with reference to themselves. Both immigrant targets were rated as significantly higher on extraversion and lower on neuroticism than the neutral target. The integrated target was rated as more open than the neutral target, and as higher than the assimilated target on neuroticism when controlling for impression management.     

The cortical organization of lexical knowledge: a dual lexicon model of spoken language processing

Current accounts of spoken language assume the existence of a lexicon where wordforms are stored and interact during spoken language perception, understanding and production. Despite the theoretical importance of the wordform lexicon, the exact localization and function of the lexicon in the broader context of language use is not well understood. This review draws on evidence from aphasia, functional imaging, neuroanatomy, laboratory phonology and behavioral results to argue for the existence of parallel lexica that facilitate different processes in the dorsal and ventral speech pathways. The dorsal lexicon, localized in the inferior parietal region including the supramarginal gyrus, serves as an interface between phonetic and articulatory representations. The ventral lexicon, localized in the posterior superior temporal sulcus and middle temporal gyrus, serves as an interface between phonetic and semantic representations. In addition to their interface roles, the two lexica contribute to the robustness of speech processing.     

Mood-congruent thoughts and personality disorder pathology

The primary aim of this research was to investigate associations that depression severity and trait anxiety have with reactions to specific, identifiable thoughts experienced by persons with varying levels of personality disorder (PD) pathology as they went about their day-to-day lives. Ninety-eight psychotropic medication-free persons participated in an experience sampling assessment procedure over four consecutive days, during which participants recorded specific thoughts experienced moments before a page signal was received and corresponding ratings of thought discomfort. Thought discomfort, regarded as an emotional reaction to thoughts or their evaluation, was moderately associated with several forms of PD pathology, particularly Cluster C pathology. Once depression severity and trait anxiety were controlled, however, associations between PD pathology and thought discomfort were eliminated. Findings from this research generally support the mood-congruence of negatively valenced thoughts in relation to PD pathology.     

Brain activation during spatial updating and attentive tracking of moving targets

Keeping aware of the locations of objects while one is moving requires the updating of spatial representations. As long as the objects are visible, attentional tracking is sufficient, but knowing where objects out of view went in relation to one's own body involves an updating of spatial working memory. Here, multiple object tracking was employed to study spatial updating and its neural correlates. In a dynamic 3D-scene, targets moved among visually indistinguishable distractors. The targets and distractors either stayed visible during continuous viewpoint changes or they turned invisible. The parametric variation of tracking load revealed load-dependent activations of the intraparietal sulcus, the superior parietal lobule, and the lateral occipital cortex in response to the attentive tracking task. Viewpoint changes with invisible objects that demanded retention and updating produced load-dependent activation only in the precuneus in line with its presumed involvement in updating spatial working memory.     

Perspective-taking and depth of theory-of-mind reasoning in sequential-move games

Theory-of-mind (ToM) involves modeling an individual's mental states to plan one's action and to anticipate others' actions through recursive reasoning that may be myopic (with limited recursion) or predictive (with full recursion). ToM recursion was examined using a series of two-player, sequential-move matrix games with a maximum of three steps. Participants were assigned the role of Player I, controlling the initial and the last step, or of Player II, controlling the second step. Appropriate for the assigned role, participants either anticipated or planned Player II's strategy at the second step, and then determined Player I's optimal strategy at the first step. Participants more readily used predictive reasoning as Player II (i.e., planning one's own move) than as Player I (i.e., anticipating an opponent's move), although they did not differ when translating reasoning outcome about the second step to optimal action in the first step. Perspective-taking influenced likelihood of predictive reasoning, but it did not affect the rate at which participants acquired it during the experimental block. We conclude that the depth of ToM recursion (related to perspective-taking mechanisms) and rational application of belief-desire to action (instrumental rationality) constitute separate cognitive processes in ToM reasoning.     

RACE/A: an architectural account of the interactions between learning, task control, and retrieval dynamics

This article discusses how sequential sampling models can be integrated in a cognitive architecture. The new theory Retrieval by Accumulating Evidence in an Architecture (RACE/A) combines the level of detail typically provided by sequential sampling models with the level of task complexity typically provided by cognitive architectures. We will use RACE/A to model data from two variants of a picture-word interference task in a psychological refractory period design. These models will demonstrate how RACE/A enables interactions between sequential sampling and long-term declarative learning, and between sequential sampling and task control. In a traditional sequential sampling model, the onset of the process within the task is unclear, as is the number of sampling processes. RACE/A provides a theoretical basis for estimating the onset of sequential sampling processes during task execution and allows for easy modeling of multiple sequential sampling processes within a task.     

Representing Spatial Structure Through Maps and Language: Lord of the Rings Encodes the Spatial Structure of Middle Earth

Spatial mental representations can be derived from linguistic and non‐linguistic sources of information. This study tested whether these representations could be formed from statistical linguistic frequencies of city names, and to what extent participants differed in their performance when they estimated spatial locations from language or maps. In a computational linguistic study, we demonstrated that co‐occurrences of cities in Tolkien’s Lord of the Rings trilogy and The Hobbit predicted the authentic longitude and latitude of those cities in Middle Earth. In a human study, we showed that human spatial estimates of the location of cities were very similar regardless of whether participants read Tolkien’s texts or memorized a map of Middle Earth. However, text‐based location estimates obtained from statistical linguistic frequencies better predicted the human text‐based estimates than the human map‐based estimates. These findings suggest that language encodes spatial structure of cities, and that human cognitive map representations can come from implicit statistical linguistic patterns, from explicit non‐linguistic perceptual information, or from both.     

Hierarchical models of simple mechanisms underlying confidence in decision making

Choice confidence is a central measure in psychological decision research, often being reported on a probabilistic scale. Simple mechanisms that describe the psychological processes underlying choice confidence, including those based on error and confirmation biases, have typically received support via fits to data averaged over subjects. While averaged data ease model development, they can also destroy important aspects of the confidence data distribution. In this paper, we develop a hierarchical model of raw confidence judgments using the beta distribution, and we implement two simple confidence mechanisms within it. We use Bayesian methods to fit the hierarchical model to data from a two-alternative confidence experiment, and we use a variety of Bayesian tools to diagnose shortcomings of the simple mechanisms that are overlooked when applied to averaged data. Bugs code for estimating the models is also supplied.