The role of ecological validity in hazard perception assessment

Hazard perception is one of the few quantifiable components of driving competency that can explain the high crash risk of novice drivers (see Horswill & McKenna, 2004). Some studies focused on investigating the stimulus part of hazard perception items by comparing several design features of the displayed traffic scenarios (e.g. Malone et al., 2012; Scialfa et al., 2013). Other empirical studies have been conducted to address the reaction part of hazard perception items (e.g. Vlakveld, 2014). Among others, existing hazard perception tests differ with respect to the ecological validity – the similarity to real-world demands – of the task that has to be completed by the participants (simple reaction, hazard localization, simulated driving, etc.). Based on the findings of general expertise research (see Glaser & Chi, 1988), the hypothesis of the present experiment was that an increased ecological validity of a hazard perception task has a positive effect on the magnitude of performance differences between more and less experienced drivers. 104 learner drivers and 51 experienced drivers were randomly assigned either to a less ecological valid version (multiple choice task) or a higher ecological valid version (reaction task) of a hazard perception test. In both conditions, the same 20 animated hazardous or non-hazardous traffic scenarios were presented. Overall, the experienced drivers outperformed the learner drivers. Principally, both of the applied versions of the test provided a criterion valid measure of driving expertise. However, increasing the ecological validity of the test by using a reaction task was especially beneficial for certain types of scenarios, leading to higher expertise-related differences. In addition, the reaction task provides response time measure as a further valid performance indicator. This allows the identification of those participants who are indeed able to detect a hazard in a certain scenario, but still have deficits in response time, compared to more experienced drivers. According to the present study, it is recommended to choose the reaction task over the multiple choice task to test hazard perception. Whether a further increase in ecological validity of the task would be useful, should be tested empirically in future research.     

Statistical Learning Is Not Affected by a Prior Bout of Physical Exercise

This study examined the effect of a prior bout of exercise on implicit cognition. Specifically, we examined whether a prior bout of moderate intensity exercise affected performance on a statistical learning task in healthy adults. A total of 42 participants were allocated to one of three conditions—a control group, a group that exercised for 15 min prior to the statistical learning task, and a group that exercised for 30 min prior to the statistical learning task. The participants in the exercise groups cycled at 60% of their respective O₂max. Each group demonstrated significant statistical learning, with similar levels of learning among the three groups. Contrary to previous research that has shown that a prior bout of exercise can affect performance on explicit cognitive tasks, the results of the current study suggest that the physiological stress induced by moderate‐intensity exercise does not affect implicit cognition as measured by statistical learning.     

Predicting Self‐Confident Behaviour with Implicit and Explicit Self‐Esteem Measures

The present research compared the validity of popular direct and indirect measures of self‐esteem in predicting self‐confident behaviour in different social situations. In line with behavioural dual‐process models, both implicit and explicit self‐esteem were hypothesized to be related to appearing self‐confident to unacquainted others. A total of 127 participants responded to the Rosenberg Self‐Esteem Scale, the Multidimensional Self‐Esteem Scale, and an adjective scale for measuring explicit self‐esteem (ESE). Participants' implicit self‐esteem (ISE) was assessed with four indirect measures: the Implicit Association Test (IAT), the name‐letter task (NLT), and two variants of an affective priming task, the reaction‐time affective priming task (RT‐APT) and the error‐based affective priming task (EB‐APT). Self‐confident behaviour was observed in four different social situations: (i) self‐introduction to a group; (ii) an ostracism experience; (iii) an interview about the ostracism experience; and (iv) an interview about one's personal life. In general, appearing self‐confident to unknown others was independently predicted by ESE and ISE. The indirect measures of self‐esteem were, as expected, not correlated, and only the self‐esteem APTs—but not the self‐esteem IAT or the NLT—predicted self‐confident behaviours. It is important to note that in particular the predictive power of the self‐esteem EB‐APT pertained to all four criteria and was incremental to the ESE measures. Copyright © 2016 European Association of Personality Psychology     

The Development and Testing of a New Version of the Cognitive Reflection Test Applying Item Response Theory (IRT)

The cognitive reflection test (CRT) is a short measure of a person's ability to resist intuitive response tendencies and to produce a normatively correct response, which is based on effortful reasoning. Although the CRT is a very popular measure, its psychometric properties have not been extensively investigated. A major limitation of the CRT is the difficulty of the items, which can lead to floor effects in populations other than highly educated adults. The present study aimed at investigating the psychometric properties of the CRT applying item response theory analyses (a two‐parameter logistic model) and at developing a new version of the scale (the CRT‐long), which is appropriate for participants with both lower and higher levels of cognitive reflection. The results demonstrated the good psychometric properties of the original, as well as the new scale. The validity of the new scale was also assessed by measuring correlations with various indicators of intelligence, numeracy, reasoning and decision‐making skills, and thinking dispositions. Moreover, we present evidence for the suitability of the new scale to be used with developmental samples. Finally, by comparing the performance of adolescents and young adults on the CRT and CRT‐long, we report the first investigation into the development of cognitive reflection. Copyright © 2015 John Wiley & Sons, Ltd.     

Test of memory malingering with children: The utility of Trial 1 and TOMMe10 as screeners of test validity

There is a growing body of research suggesting that the shorter versions of the Test of Memory Malingering (TOMM) may provide an accurate assessment of effort in children. During neuropsychological evaluations, some circumstances result in only one completed trial of the TOMM or partial completion of a trial. Research suggests that a cut-score of 40 or 41 on Trial1 is highly predictive of passing the TOMM overall. In the current study, 194 school-age children with academic and/or behavioral problems were used to compare the accuracy of TOMM1 and TOMMe10 (errors on the first 10 items of TOMM1) in predicting passing/failing of TOMM2. For the children in this sample, a score of < 40 items correct (≥ 10 errors) on TOMM1 was highly accurate in predicting a passing performance on the TOMM2 (sensitivity = .80, specificity = .91) with a Negative Predictive Value = .98 at the malingering base rate of 7% (TOMM2 failure in our sample). A score of 2 errors (8 items correct) on the TOMMe10 was slightly less sensitive than that of the TOMM1 (specificity = .96, sensitivity = .53) but with a similar Negative Predictive Value (.96). Consistent with the research from adult populations, TOMM1 and TOMMe10 appear to be quite accurate in predicting performance on the standard administration of the TOMM and may be useful screeners. However, compared to that found in adult samples, slight differences in suggested cutoffs for TOMM1 and TOMMe10 may be warranted for children.     

The International Test Commission Guidelines on the Security of Tests,Examinations, and Other Assessments

The amount and severity of security threats have increased considerably over the past two decades, calling into question the validity of assessments administered around the world. These threats have increased for a number of reasons, including the popular use of computerized and online technologies for test administration and the use of almost undetectable technologies for capturing test content and illegally sharing it instantly across borders and cultures. No assessment program, large or small, is immune to this potential damage. The International Test Commission has recognized the critical need for every organization with an important assessment program to be aware of these and be prepared to counter them. It was for this purpose that these guidelines were developed. Knowing the threats and the guidelines will lead to effective measures to protect the program and its assets, maintaining the value of the tests and assessments to the international community. The guidelines listed in this article provide recommendations on planning for better security, maintaining security during the development of tests and while they are administered, and responding well when a security breach occurs. Following these guidelines will create a significant protective barrier between those who willingly commit test fraud and the valuable assets a program has spent time and money to build.     

Psychometrics in Support of a Valid Assessment of Linguistic Minorities: Implications for the Test and Sampling Designs

In this study, we propose that the unique needs and characteristics of linguistic minorities should be considered throughout the test development process. Unlike most measurement invariance investigations in the assessment of linguistic minorities, which typically are conducted after test administration, we propose strategies that focus on the early stages of test development. Our approaches involve considering linguistic minorities in the selection of the test and sampling designs. We posit that joint consideration of these issues preemptively can strengthen the claims derived from tests used to assess linguistically diverse populations. This also will contribute to more psychometrically robust assessments, which can yield accurate and valid score-based inferences for linguistic minorities. To this end, we provide various examples and methodological approaches that can be used in the selection of the test and sampling designs that take these issues into consideration.     

Speech segmentation by statistical learning is supported by domain-general processes within working memory

The purpose of this study was to examine the extent to which working memory resources are recruited during statistical learning (SL). Participants were asked to identify novel words in an artificial speech stream where the transitional probabilities between syllables provided the only segmentation cue. Experiments 1 and 2 demonstrated that segmentation performance improved when the speech rate was slowed down, suggesting that SL is supported by some form of active processing or maintenance mechanism that operates more effectively under slower presentation rates. In Experiment 3 we investigated the nature of this mechanism by asking participants to perform a two-back task while listening to the speech stream. Half of the participants performed a two-back rhyme task designed to engage phonological processing, whereas the other half performed a comparable two-back task on un-nameable visual shapes. It was hypothesized that if SL is dependent only upon domain-specific processes (i.e., phonological rehearsal), the rhyme task should impair speech segmentation performance more than the shape task. However, the two loads were equally disruptive to learning, as they both eradicated the benefit provided by the slow rate. These results suggest that SL is supported by working-memory processes that rely on domain-general resources.     

Disentangling proactive and reactive aggression in children using self-report

The distinction between proactive and reactive functions of aggression is one of the most common divisions when investigating aggression among children and adolescents. To date, self-report is the least used measurement, despite existing literature supporting the view that the best informant regarding internal processes and motives are children themselves. The main aim of this study was to examine the construct and concurrent validity of a new self-report questionnaire, which aims to disentangle acts of reactive vs. proactive aggression that are most common within the daily lives of children. We examined the self-report measure among 578 children (313 girls, 265 boys, mean age 11 years, range 9–13 years). Most children (90% boys; 85% girls) reported at least one act of aggression over the last four weeks. Furthermore, the outcomes support the two-factor structure (reactive and proactive aggression) and the questionnaire showed good concurrent and discriminant validity with measures for emotional and social functioning. This study validates the use of the self-report instrument for reactive and proactive aggression and demonstrates that children can successfully distinguish between their own motives for reactive and proactive forms of aggressive behaviours.     

Construct validity and construct equivalence of work values across boys and girls

This study investigated the structural validity and construct comparability, across boys and girls, of a newly developed measure of work values. The total sample was split into two equal parts, and based on an item-level factor analysis of the first sub-sample, fit statistics provided support for a six-factor first-order structural model with the following constructs: Power, Money, Tidy environment, Teamwork, Work–Family Balance and Ability Utilization. This structure was confirmed in the second sub-sample, and strict measurement invariance for the measurement model was obtained in this sample. Additionally, the factor variances/covariances were invariant. Five non-invariant latent means were found, with girls scoring higher on the Balance and Personal Challenge values, and boys scoring higher on the Power, Money and Teamwork values.