Does the cognitive reflection test measure cognitive reflection? A mathematical modeling approach

We used a mathematical modeling approach, based on a sample of 2,019 participants, to better understand what the cognitive reflection test (CRT; Frederick In Journal of Economic Perspectives, 19, 25–42, 2005) measures. This test, which is typically completed in less than 10 min, contains three problems and aims to measure the ability or disposition to resist reporting the response that first comes to mind. However, since the test contains three mathematically based problems, it is possible that the test only measures mathematical abilities, and not cognitive reflection. We found that the models that included an inhibition parameter (i.e., the probability of inhibiting an intuitive response), as well as a mathematical parameter (i.e., the probability of using an adequate mathematical procedure), fitted the data better than a model that only included a mathematical parameter. We also found that the inhibition parameter in males is best explained by both rational thinking ability and the disposition toward actively open-minded thinking, whereas in females this parameter was better explained by rational thinking only. With these findings, this study contributes to the understanding of the processes involved in solving the CRT, and will be particularly useful for researchers who are considering using this test in their research.     

Is amodal completion necessary for the formation of illusory figures?

Kanizsa's hypothesis suggests that the creation of an anomalous surface is due to the amodal completion of the inducers. In the present paper a new pattern that is able to disconfirm this explanation is presented. According to the Helmholtz-Ratoosh law amodal completion only occurs when the borders of two adjacent surfaces meet forming T-shaped junctions. When the borders of the two adjacent surfaces have Y-shaped junctions, amodal completion is absent. However, when a pattern inducing an anomalous figure has the latter figural characteristics, in spite of the absence of amodal completion, an illusory figure is still visible. In this paper a set of experimental results (carried out by means of a magnitude estimation procedure as well as the method of constant stimuli) supporting the aforementioned observations is presented.     

Is the learn unit a fundamental measure of pedagogy?

We propose a measure of teaching, the learn unit, that explicitly describes the interaction between teachers and their students. The theoretical, educational research, and applied behavior analysis literatures all converge on the learn unit as a fundamental measure of teaching. The theoretical literature proposes the construct of the interlocking operant and embraces verbal behavior, social interaction, and translations of psychological constructs into complex theoretical respondent-operant interactions and behavior-behavior relations. Research findings in education and applied behavior analysis on engaged academic time, opportunity to respond, active student responding, teacher-student responding, student-teacher responding, tutor-tutee responding, tutee-tutor responding, and verbal episodes between individuals all support a measure of interlocking responses. More recently, research analyzing the components of both the students' and teachers' behavior suggests that the learn unit is the strongest predictor of effective teaching. Finally, we propose applications of the learn unit to other issues in pedagogy not yet researched and the relation of learn units to the verbal behavior of students.     

Is one item an adequate measure of cognitive learning?

While the "leaning loss" scale is a valid means of estimating cognitive learning, the test is valid only when used as designed-with two items.     

Is the area measure a historical anomaly?

Green's well-known area theorem establishes an equivalence between the area under the yes-no ROC curve and the percent correct of an unbiased observer in a two-alternative forced-choice (2AFC) task with equivalent stimuli. In this article, we show that this conversion from yes-no detection data to hypothetical performance in a 2AFC task is unnecessary: The same yes-no detection data that are used to compute the area statistic can always be used to compute the percent correct of an unbiased observer in the yes-no detection task itself. We also show that the ROC curve may not be the ideal graphical device for many investigators. A more natural representation of the difficulty of a discrimination task is obtained by plotting the distribution of the posterior betting odds under equal base rates, which can be estimated from their distributions under unequal base rates. Finally, unlike the area measure and other traditional detection theory statistics, both the yes-no percent correct measure and the odds distributions generalize in an obvious and direct way to classification paradigms with more than two responses (e.g., identification).     

What does the implicit association test measure? A test of the convergent and discriminant validity of prejudice-related IATs

Drawing on recent criticism of the Implicit Association Test (IAT), the present study tested the convergent and discriminant validity of two prejudice-related IATs to corresponding explicit prejudice measures in a German student sample (N = 61). Confirming convergent validity, (a) an IAT designed to assess negative associations related to Turkish people was significantly related to the explicit endorsement of prejudiced beliefs about Turkish people, and (b) an IAT designed to assess negative associations related to East Asians was significantly related to explicit prejudice against East Asians. Moreover, confirming discriminant validity, (c) the Asian IAT was unrelated to the explicit endorsement of prejudiced beliefs about Turkish people, and (d) the Turkish IAT was unrelated to explicit prejudice against Asian people. These results further corroborate the assumption that the IAT is a valid method to assess the strength of evaluative associations in the domain of prejudice and stereotypes.     

Is there still a cognitive cost of being a twin in the UK?

ObjectiveTo investigate whether twins still perform below singletons in cognitive ability tests and in national educational attainment examinations in a recent UK cohort of 11-year-olds.DesignPopulation-based cohort study, using a validated method of twin extraction.SettingEnglish state schools.Participants178,599 pupils (mean age 11 years) attending English state schools in 2004, representing 93% of local educational authorities.Main outcome measuresVerbal, quantitative and nonverbal reasoning measured by the Cognitive Abilities Test — Third Edition (CAT3), and educational attainment in English, maths and science from Key Stage 2 national curriculum tests.ResultsTwin-singleton differences in cognitive ability were of negligible effect size. Mean performance of twins was 1% of a SD below singletons on general cognitive ability, 3% lower on verbal reasoning, and less than 1% of a SD lower on quantitative and nonverbal reasoning, with and without adjustment for sex. Almost identical proportions of twins and singletons attained the expected standards of educational performance in English (80.4% versus 80.5%, respectively), maths (77.4% versus 77.3%), and science (90.1% versus 88.8%). There were no differences in rates of twins and singletons achieving higher or lower performance levels.ConclusionThe cognitive cost of being a twin reported in previous cohorts of UK children no longer exists in a large, contemporary and population-representative sample.     

Is the DOSPERT gender invariant? A psychometric test of measurement invariance

It is well established in the risk literature that men tend to take more risks than women. This gender difference, however, is often qualified by its domain specificity. Considering recent research on the domain generality of risk taking as a disposition, there is a need to examine the degree to which men take more risks than women, in general. In order to make substantive conclusions about the gender differences in risk‐taking propensity, one must first establish measurement invariance, which is required for the meaningful interpretation of observed group differences. In this paper, we examined the measurement invariance of the Domain‐Specific Risk‐Taking scale (DOSPERT)—one of the most popular measures of individual differences in risk taking. We found that the DOSPERT violated configural invariance in a bifactor model, indicating that the underlying factor structure of the DOSPERT differs between men and women. Even after removing the social risk dimension, DOSPERT still failed to reach scalar invariance. Taken together, these findings suggest that score differences in the DOSPERT may be due to response artifacts rather than true differences in the latent construct. Therefore, gender differences in the DOSPERT must be interpreted with caution. Implications for the measurement of risk taking are discussed.     

Is EEG α a consistent measure of individual differences?

Integrated EEG activity in the 9–13 Hz band was recorded from 24 Ss under three experimental conditions (Relaxation, Attentive Listening and Mental Arithmetic) in two separate sessions. Analysis of components of variance due to persons and the interactions of persons with sessions and with conditions pointed to the importance of the persons × conditions interaction. It is concluded that under typically employed conditions of testing EEG α is not a consistent measure of individual differences.     

Epistemic action: A measure for cognitive support in tangible user interfaces?

The quality of user interfaces is often measured in terms of efficiency, effectiveness, and satisfaction. In the area of tangible user interfaces, epistemic—or exploratory—action has been suggested as a fourth measure of quality. In computer game studies (Kirsh & Maglio, 1992, 1994), players used epistemic actions to modify the environment, which helped them determine the correct position of blocks with less mental effort. There, the researchers found that it might be easier to physically modify the external world and then interpret it than to compute and interpret a new state mentally. Specifically, epistemic action may be a relevant concept when researching tangible user interfaces incorporating physical handles. This article examines the potential relations between the three traditional measures of usability and epistemic actions using three spatial planning tools with different degrees of physicality. The results indicate that epistemic action is a measure that is independent of the three traditional usability measures: efficiency, effectiveness, and satisfaction. However, epistemic action does not increase linearly with the physicality of a user interface, and it probably is a more complex measure that is also related to the reusability of the interface. Further research is needed to fully understand the potential of this measure.     

A test of Vitz and Todd’s model of the judged complexity of straight-line figures

Two sets of stimuli, each consisting of nine geometric patterns, were computer-generated: in one set, the patterns were squares which were partitioned randomly into rectangles; in the other, the patterns were squares which were partitioned randomly into polygons. Ten subjects gave magnitude estimates of the complexity of the stimuli. The Vitz and Todd (1971) index of complexity was not a consistently better predictor of estimated complexity than the ratio of the number of lines in each pattern to an index of symmetry. Moreover, the power function relating the complexity estimates of the rectangular partitions to the Vitz and Todd index was significantly different from the power function relating the complexity of the polygonal partitions to the Vitz and Todd index. It is concluded that the Vitz and Todd model requires major additional assumptions in order to be a candidate for a process model of pattern perception.     

Is a modular cognitive architecture compatible with the direct perception of mental states?

The Direct Social Perception Hypothesis maintains that we can perceive other people’s psychological states. Furthermore, it claims that doing so does not require any cognitive process that is simulative or theory-like, putting it in sharp contrast with mainstream accounts of social cognition. This paper contrasts the DSPH against the modular account of mindreading as proposed by Peter Carruthers and H. Clark Barrett. It maintains that the modularity view can respond to the challenges levelled by the DSPH, and that the positions are not as distinct as they originally appear. Finally, the paper discusses the role of non-folk psychological state concepts in our perceptions of other people.     

Is state-trace analysis an appropriate tool for assessing the number of cognitive systems?

There is now much evidence that humans have multiple memory systems, and evidence is also building that other cognitive processes are mediated by multiple systems. Even so, several recent articles have questioned the existence of multiple cognitive systems, and a number of these have based their arguments on results from state-trace analysis. State-trace analysis was not developed for this purpose but, rather, to identify data sets that are consistent with variation in a single parameter. All previous applications have assumed that state-trace plots in which the data fall on separate curves rule out any model in which only a single parameter varies across the two tasks under study. Unfortunately, this assumption is incorrect. Models in which only one parameter varies can generate any type of state-trace plot, as can models in which two or more parameters vary. In addition, it is straightforward to show that both single-system and multiple-systems models can generate state-trace plots that are considered in the literature to be consistent with either one or multiple cognitive systems. Thus, without additional information, there is no empirical state-trace plot that supports any inferences about the number of underlying parameters or systems.     

Is probability matching smart? Associations between probabilistic choices and cognitive ability

In three experiments involving over 1,500 university students (n = 1,557) and two different probabilistic choice tasks, we found that the utility-maximizing strategy of choosing the most probable alternative was not the majority response. In a story problem version of a probabilistic choice task in which participants chose from among five different strategies,the maximizing response and the probability-matching response were each selected by a similar number of students (roughly 35% of the sample selected each). In a more continuous, or trial-by-trial, task, the utility-maximizing response was chosen by only one half as many students asthe probability-matching response. More important, in both versions of the task, the participants preferring the utility-maximizing response were significantly higher in cognitive ability than were the participants showing a probability-matching tendency. Critiques of the traditional interpretation of probability matching as nonoptimal may well help explain why some humans are drawn to the nonmaximizing behavior of probability matching, but the traditional heuristics and biases interpretation can most easily accommodate the finding that participants high in computational ability are more likely to carry out the rule-based cognitive procedures that lead to maximizing behavior.     

Is running a state of mind? Sports training as a potential method for developing cognitive flexibility

The study aimed to test the significance of sports participation as a potential means of improving cognitive function, particularly cognitive flexibility. Based on the characteristics of orienteering, such as frequent changes of behavioural strategies in response to changes in the situation or the simultaneous performance of several mental activities, we assumed that practising this sport could foster the development of cognitive flexibility. Two groups of volunteers were compared: 50 middle and long-distance runners and 50 orienteering runners in terms of their performance on the following measures of cognitive flexibility: a divergent thinking task, a computer flexibility task, Cognitive Flexibility Scale, and Verbal Fluency Test as a measure of executive function. Orienteering runners outperformed others on all measures except the Cognitive Flexibility Scale. Furthermore, we found that training characteristics (regularity, frequency, participation in competitions) were associated with levels of cognitive flexibility, particularly among orienteering runners, where they explained between 38% and 39% of the overall flexibility variance. Our results suggest that cognitive flexibility can be developed through sports training requiring effective dealing in a changing, complex environment. We also discuss the implications of our results for cognitive training research.