Investigating Test-Taking Behaviors Using Timing and Process Data

The use of computer-based assessments makes the collection of detailed data that capture examinees’ progress in the tests and time spent on individual actions possible. This article presents a study using process and timing data to aid understanding of an international language assessment and the examinees. Issues regarding test-taking strategies, test speededness, test design, and their relationship to examinees’ demographic backgrounds and performance are also discussed.     

First Among Equals: Hybridization of Cognitive Diagnostic Assessment and Evidence-Centered Game Design

The objective of the present article is to explore differences and similarities between cognitive diagnostic assessment (CDA) and evidence-centered game design (ECgD) in the service of intentional hybridization. Although some testing specialists might argue that both are essentially the same given their origins in principled assessment design and equivalency of measurement models, this view misses differences in their focus and operationalization. Given the strengths of both CDA and ECgD, there is motivation to consider ways in which each can deliberately inform the other. The intentional hybridization of CDA and ECgD has, at least in principle, significant advantages to produce a stronger offspring than either parent alone. The article includes four sections: (1) conceptual differences between CDA and ECgD, (2) conceptual similarities between CDA and ECgD, (3) challenges with CDA and ECgD, including narrowness of cognitive models, fidelity with learning, ocean of data, sensitivity to diverse learners, reliance on multidimensional psychometric models, and how hybridization may help, and (4) implications for educational assessment in the twenty-first century around the globe.     

人格特质对考试焦虑的影响:多重中介效应分析

目的:探讨自尊、惧怕负面评价及无法忍受不确定性在人格特质和考试焦虑间的中介作用。方法:采用人格问卷简式量表EPQ-RSC、Rosenberg自尊问卷、惧怕负面评价量表简表、无法忍受不确定性量表和考试焦虑量表测量410名在校高中生和大学生。结果:(1)相关分析表明,神经质、惧怕负面评价、无法忍受不确定性及考试焦虑之间彼此正相关显著;外向性与自尊正相关显著,与惧怕负面评价、无法忍受不确定性及考试焦虑等负相关显著;精神质与自尊、惧怕负面评价及考试焦虑等负相关显著;自尊与神经质、惧怕负面评价、无法忍受不确定性及考试焦虑负相关显著。(2)回归分析表明,人格特质中的神经质和精神质能显著预测考试焦虑,无法忍受不确定性在人格特质和考试焦虑之间起部分中介作用;结构方程模型分析结果显示,人格特质中神经质和精神质对考试焦虑的直接效应显著,经由自尊、惧怕负面评价和无法忍受不确定性对考试焦虑的间接效应也极为显著。结论:自尊、惧怕负面评价和无法忍受不确定性在人格特质与考试焦虑间起着中介作用。     

A closer look at the discrimination outcomes in the IAT literature

To what extent the IAT (Implicit Association Test, Greenwald et al., 1998) predicts racial and ethnic discrimination is a heavily debated issue. The latest meta‐analysis by Oswald et al. (2013) suggests a very weak association. In the present meta‐analysis, we switched the focus from the predictor to the criterion, by taking a closer look at the discrimination outcomes. We discovered that many of these outcomes were not actually operationalizations of discrimination, but rather of other related, but distinct, concepts, such as brain activity and voting preferences. When we meta‐analyzed the main effects of discrimination among the remaining discrimination outcomes, the overall effect was close to zero and highly inconsistent across studies. Taken together, it is doubtful whether the amalgamation of these outcomes is relevant criteria for assessing the IAT's predictive validity of discrimination. Accordingly, there is also little evidence that the IAT can meaningfully predict discrimination, and we thus strongly caution against any practical applications of the IAT that rest on this assumption. However, provided that the application is thoroughly informed by the current state of the literature, we believe the IAT can still be a useful tool for researchers, educators, managers, and students who are interested in attitudes, prejudices, stereotypes, and discrimination.     

Improving Psychological Measurement: Does It Make a Difference? A Comment on Nesselroade and Molenaar (2016)

Nesselroade and Molenaar advocate the use of an idiographic filter approach. This is a fixed-effects approach, which may limit the number of individuals that can be simultaneously modeled, and it is not clear how to model the presence of subpopulations. Most important, Nesselroade and Molenaar's proposal appears to be best suited for modeling long time series on a few variables for a few individuals. Long time series are not common in psychological applications. Can it be applied to the usual longitudinal data we face? These are characterized by short time series (four to five points in time), hundreds of individuals, and dozens of variables. If so, what do we gain? Applied settings most often involve between-individual decisions. I conjecture that their approach will not outperform common, simpler, methods. However, when intraindividual decisions are involved, their approach may have an edge.     

Properties of bootstrap tests for N‐of‐1 studies

N‐of‐1 study designs involve the collection and analysis of repeated measures data from an individual not using an intervention and using an intervention. This study explores the use of semi‐parametric and parametric bootstrap tests in the analysis of N‐of‐1 studies under a single time series framework in the presence of autocorrelation. When the Type I error rates of bootstrap tests are compared to Wald tests, our results show that the bootstrap tests have more desirable properties. We compare the results for normally distributed errors with those for contaminated normally distributed errors and find that, except when there is relatively large autocorrelation, there is little difference between the power of the parametric and semi‐parametric bootstrap tests. We also experiment with two intervention designs: ABAB and AB, and show the ABAB design has more power. The results provide guidelines for designing N‐of‐1 studies, in the sense of how many observations and how many intervention changes are needed to achieve a certain level of power and which test should be performed.     

Attentional orienting and executive control are affected by different types of meditation practice

Several studies have demonstrated the beneficial effects of meditation on attention. The present study investigated the relationship between focused attention (FA) and open monitoring (OM) meditation skills and the various functions of attention. In Experiment 1, we executed the attention network test and compared the performance of experts on dandao meditation with that of ordinary people on this test. The results indicated that the experts specializing in OM meditation demonstrated greater attentional orienting ability compared with those specializing in FA meditation and the control group. In addition, both expert groups registered improvements in their executive control abilities compared with the control group. In Experiment 2, we trained beginners in FA meditation for 3 months. The results showed that the experimental group exhibited significantly enhanced executive control ability. We infer that FA meditation skills promote executive control function and OM meditation skills promote both executive control and attentional orienting functions.     

Religiousness,religious coping methods,and cognitive test anxiety among African-American college students

This study explored the roles of religiousness and religious coping methods in predicting cognitive test anxiety. A convenience sample of 121 African-American students (97 females and 24 males) ranging in age from 18 to 39 (M_age?=?20.16), attending a historically Black university completed an online questionnaire assessing demographic information, religiousness, religious coping methods, and cognitive test anxiety. Results showed that negative religious coping methods were significant factors in predicting cognitive test anxiety. These relationships may be pertinent for understanding salient factors that influence cognitive test anxiety in African-American college students.     

An investigation of cognitive test performance across conditions of silence,background noise and music as a function of neuroticism

The present study investigates the role of trait neuroticism on cognitive performance under distraction. Seventy participants were given a personality test and then undertook a number of different cognitive tasks in silence, in the presence of popular music and in background noise. It was predicted that performance on a general intelligence test, a test of abstract reasoning, and a mental arithmetic task would be adversely affected by background sounds. It was predicted that neuroticism would be negatively correlated with performance on the mental arithmetic task but only when the individuals were working in the presence of background sound. Stable vs. unstable participant's performance on a mental arithmetic task during noise was significantly higher as predicted. The results provided partial support for the hypotheses and are discussed with respect to previous findings in the literature on personality (particularly introversion–extraversion) and distraction on cognitive task performance. Limitations are noted.