Developing new online calibration methods for multidimensional computerized adaptive testing

Multidimensional computerized adaptive testing (MCAT) has received increasing attention over the past few years in educational measurement. Like all other formats of CAT, item replenishment is an essential part of MCAT for its item bank maintenance and management, which governs retiring overexposed or obsolete items over time and replacing them with new ones. Moreover, calibration precision of the new items will directly affect the estimation accuracy of examinees’ ability vectors. In unidimensional CAT (UCAT) and cognitive diagnostic CAT, online calibration techniques have been developed to effectively calibrate new items. However, there has been very little discussion of online calibration in MCAT in the literature. Thus, this paper proposes new online calibration methods for MCAT based upon some popular methods used in UCAT. Three representative methods, Method A, the ‘one EM cycle’ method and the ‘multiple EM cycles’ method, are generalized to MCAT. Three simulation studies were conducted to compare the three new methods by manipulating three factors (test length, item bank design, and level of correlation between coordinate dimensions). The results showed that all the new methods were able to recover the item parameters accurately, and the adaptive online calibration designs showed some improvements compared to the random design under most conditions.     

Non‐ignorable missingness item response theory models for choice effects in examinee‐selected items

Examinee‐selected item (ESI) design, in which examinees are required to respond to a fixed number of items in a given set, always yields incomplete data (i.e., when only the selected items are answered, data are missing for the others) that are likely non‐ignorable in likelihood inference. Standard item response theory (IRT) models become infeasible when ESI data are missing not at random (MNAR). To solve this problem, the authors propose a two‐dimensional IRT model that posits one unidimensional IRT model for observed data and another for nominal selection patterns. The two latent variables are assumed to follow a bivariate normal distribution. In this study, the mirt freeware package was adopted to estimate parameters. The authors conduct an experiment to demonstrate that ESI data are often non‐ignorable and to determine how to apply the new model to the data collected. Two follow‐up simulation studies are conducted to assess the parameter recovery of the new model and the consequences for parameter estimation of ignoring MNAR data. The results of the two simulation studies indicate good parameter recovery of the new model and poor parameter recovery when non‐ignorable missing data were mistakenly treated as ignorable.     

Estimating Between-Person and Within-Person Subscore Reliability with Profile Analysis

Subscores are of increasing interest in educational and psychological testing due to their diagnostic function for evaluating examinees' strengths and weaknesses within particular domains of knowledge. Previous studies about the utility of subscores have mostly focused on the overall reliability of individual subscores and ignored the fact that subscores should be distinct and have added value over the total score. This study introduces a profile reliability approach that partitions the overall subscore reliability into within-person and between-person subscore reliability. The estimation of between-person reliability and within-person reliability coefficients is demonstrated using subscores from number-correct scoring, unidimensional and multidimensional item response theory scoring, and augmented scoring approaches via a simulation study and a real data study. The effects of various testing conditions, such as subtest length, correlations among subscores, and the number of subtests, are examined. Results indicate that there is a substantial trade-off between within-person and between-person reliability of subscores. Profile reliability coefficients can be useful in determining the extent to which subscores provide distinct and reliable information under various testing conditions.     

Scrutinizing the Emotional Nature of Intuitive Coherence Judgments

Dual‐system models propose that cognitive processing can occur either intuitively or deliberately. Unlike deliberate decision strategies, intuitive ones are assumed to have an emotional component attached to the decision process. We tested if intuitive decisions are indeed accompanied by an emotional response while deliberate decisions are not. Specifically, we conducted a psychophysiological study in which participants were instructed to decide either intuitively or deliberately if three simultaneously presented words were semantically coherent or incoherent (triad task). The degree of emotionality of these two decision strategies (intuitive vs. deliberate) was compared using changes in electrodermal activity (EDA) and the reaction time (RT) effect of an affective priming paradigm as primary measurements. Based on a valence‐arousal model, our results revealed that intuitive and deliberate judgments do not differ as to their emotional valence but that they do differ in emotional arousal. Most notably, sympathetic activation during intuitive judgments was significantly lower compared to sympathetic activation during deliberate judgments. Our results reflect that a relaxed state of mind—manifested in low sympathetic activity—could underlie the holistic processing that is assumed to facilitate the proliferation of semantic associations during coherence judgments. This suggests that coherence judgments made under an (instructed) intuitive decision mode have a specific psychophysiological signature and that arousal is the differentiating component between intuitive and deliberate decision strategies. © 2016 The Authors Journal of Behavioral Decision Making Published by John Wiley & Sons Ltd.     

Effects of interaction pattern on family harmony and well-being: Test of interpersonal theory, Relational-Models theory, and Confucian ethics

Two studies, involving 242 and 245 families, respectively, were conducted to explore the effects of family interaction on family harmony and well-being. Hypotheses were drawn from interpersonal theory, Relational-Models theory, and Confucian ethics. Each of the four members in a family, including father, mother, son, and daughter, completed a questionnaire designed to assess their behaviors toward each of the other three members. Six categories of interpersonal behavior were measured, including love, directing, domineering, hostility, submission, and respect. Both studies found that family affective interaction pattern following the Communal Sharing model contributed to family harmony and well-being. For each family relationship, the existence of particular forms of status differential (Authority Ranking) was beneficial, and several factors, such as role expectation and complementarity, moderated the effects of the status differential of a particular relationship. In general, positive effects were associated with interaction patterns which were consistent with Confucian ethics.     

THEOLOGY AND PHYSICS FORTY YEARS LATER

Abstract. Almost forty years later I look back on a 1966 article on theology and physics by Sanford Brown and my response published with it. I reflect on his hope that theological seminaries would give attention to the methods used in scientific inquiry. I compare our comments with subsequent thought on three issues: (1) the role of models in science and religion; (2) the relation of wholes to parts in physics and other sciences and the debate over reductionism and emergence; and (3) the implications of quantum physics for theology, including the possibility of divine action at the quantum level.     

Explaining after by before: Basic aspects of a dynamic systems approach to the study of development

The basic properties of a dynamic systems approach of development are illustrated by contrasting two simple equations. One, y_t+1 = f (y_t), is characteristic of dynamic systems models. The other, y_i = f (x_i), refers to what, for the sake of simplicity, is referred to as the standard developmental approach. We give illustrations from cognitive, language and social development to show the characteristic differences of these two types of models and show their complementarity. The article further compares the “Bloomington” with the “Groningen” approach to dynamic systems theorizing in developmental psychology. It continues with a discussion of two important questions. One involves the issue of measurement and the nature of developmental variables from the viewpoint of dynamic systems. The second concerns the question of short- and long-term time scales in developmental models, which is discussed on the basis of an example, namely dyadic interaction of young children in the context of different social statuses.     

Normal Personality Correlates of Chronic Pain Subgroups

A heterogeneous sample of chronic pain patients was first classified into one of three previously identified subgroups (i.e., dysfunctional, interpersonally distressed, adaptive) using the Multidimensional Pain Inventory (R. D. Kerns, D. C. Turk, & T. E. Rudy, 1985). Multiple differences at the domain and facet levels of the Revised NEO PI-R Personality Inventory (P. T. Costa, Jr. & R. R. McCrae, 1992) were found among the three subgroups of patients. In particular, the interpersonally distressed patients showed a greater degree of Neuroticism and were more introverted. Further analyses revealed that this group had tendencies toward feeling more vulnerable, being less trustful, and experiencing less positive emotions than the other groups. The dysfunctional group of patients was notable in terms of their low activity level and lower Openness (to new experience). Adaptive copers reported being emotionally stable, having a moderate degree of Extraversion, and being open to a wider variety of experiences. These differences may act as mediating influences in differing patient responsiveness to pain management programs.     

Phonological abstraction in processing lexical-tone variation: evidence from a learning paradigm

There is a growing consensus that the mental lexicon contains both abstract and word-specific acoustic information. To investigate their relative importance for word recognition, we tested to what extent perceptual learning is word specific or generalizable to other words. In an exposure phase, participants were divided into two groups; each group was semantically biased to interpret an ambiguous Mandarin tone contour as either tone1 or tone2. In a subsequent test phase, the perception of ambiguous contours was dependent on the exposure phase: Participants who heard ambiguous contours as tone1 during exposure were more likely to perceive ambiguous contours as tone1 than participants who heard ambiguous contours as tone2 during exposure. This learning effect was only slightly larger for previously encountered than for not previously encountered words. The results speak for an architecture with prelexical analysis of phonological categories to achieve both lexical access and episodic storage of exemplars.     

TILLICHIAN TELEODYNAMICS: AN EXAMINATION OF THE MULTIDIMENSIONAL UNITY OF EMERGENT LIFE

Abstract Emergence theory has generated many significant new questions for dialogue between theology and science. My work will examine the models of one emergence theorist, Terrence Deacon, and consider the constructive potential of Tillich's multidimensional unity of life for responding to the theological ramifications of this account of emergence theory. Such a Tillich‐inspired constructive process will rely upon Robert Russell's method of “Creative Mutual Interaction.” Building on the interactive quality of Russell's method, I will also begin to offer suggestions for how Tillich's theological themes might influence scientific research programs using Deacon's emergence theory by contributing to the process of defining life. Finally, I will conclude by identifying three facets of continued research that stem from this analysis, focusing primarily on its implications for theological anthropology and what it means to be in the image of God.