Moving in Parallel Toward a Modern Modeling Epistemology: Bayes Factors and Frequentist Modeling Methods

The Bayesian-frequentist debate typically portrays these statistical perspectives as opposing views. However, both Bayesian and frequentist statisticians have expanded their epistemological basis away from a singular focus on the null hypothesis, to a broader perspective involving the development and comparison of competing statistical/mathematical models. For frequentists, statistical developments such as structural equation modeling and multilevel modeling have facilitated this transition. For Bayesians, the Bayes factor has facilitated this transition. The Bayes factor is treated in articles within this issue of Multivariate Behavioral Research. The current presentation provides brief commentary on those articles and more extended discussion of the transition toward a modern modeling epistemology. In certain respects, Bayesians and frequentists share common goals.     

Psychometric properties of the General Health Questionnaire-12 in a sample of Hong Kong employees

The General Health Questionnaire-12 (GHQ-12) is a widely used instrument for measuring psychological strain, but the factor structure of the GHQ-12 is inconclusive. The present study examined one-factor, two- and three-factor models of the GHQ-12 using structural equations modelling in a longitudinal data-set of Hong Kong employees. The findings supported a two-factor model consisting of a ‘Social Dysfunction’ factor measured by three items, and an ‘Anxiety/Depression’ factor measured by four items. Implications and limitations are discussed.     

The role of ecological validity in hazard perception assessment

Hazard perception is one of the few quantifiable components of driving competency that can explain the high crash risk of novice drivers (see Horswill & McKenna, 2004). Some studies focused on investigating the stimulus part of hazard perception items by comparing several design features of the displayed traffic scenarios (e.g. Malone et al., 2012; Scialfa et al., 2013). Other empirical studies have been conducted to address the reaction part of hazard perception items (e.g. Vlakveld, 2014). Among others, existing hazard perception tests differ with respect to the ecological validity – the similarity to real-world demands – of the task that has to be completed by the participants (simple reaction, hazard localization, simulated driving, etc.). Based on the findings of general expertise research (see Glaser & Chi, 1988), the hypothesis of the present experiment was that an increased ecological validity of a hazard perception task has a positive effect on the magnitude of performance differences between more and less experienced drivers. 104 learner drivers and 51 experienced drivers were randomly assigned either to a less ecological valid version (multiple choice task) or a higher ecological valid version (reaction task) of a hazard perception test. In both conditions, the same 20 animated hazardous or non-hazardous traffic scenarios were presented. Overall, the experienced drivers outperformed the learner drivers. Principally, both of the applied versions of the test provided a criterion valid measure of driving expertise. However, increasing the ecological validity of the test by using a reaction task was especially beneficial for certain types of scenarios, leading to higher expertise-related differences. In addition, the reaction task provides response time measure as a further valid performance indicator. This allows the identification of those participants who are indeed able to detect a hazard in a certain scenario, but still have deficits in response time, compared to more experienced drivers. According to the present study, it is recommended to choose the reaction task over the multiple choice task to test hazard perception. Whether a further increase in ecological validity of the task would be useful, should be tested empirically in future research.     

The Development and Testing of a New Version of the Cognitive Reflection Test Applying Item Response Theory (IRT)

The cognitive reflection test (CRT) is a short measure of a person's ability to resist intuitive response tendencies and to produce a normatively correct response, which is based on effortful reasoning. Although the CRT is a very popular measure, its psychometric properties have not been extensively investigated. A major limitation of the CRT is the difficulty of the items, which can lead to floor effects in populations other than highly educated adults. The present study aimed at investigating the psychometric properties of the CRT applying item response theory analyses (a two‐parameter logistic model) and at developing a new version of the scale (the CRT‐long), which is appropriate for participants with both lower and higher levels of cognitive reflection. The results demonstrated the good psychometric properties of the original, as well as the new scale. The validity of the new scale was also assessed by measuring correlations with various indicators of intelligence, numeracy, reasoning and decision‐making skills, and thinking dispositions. Moreover, we present evidence for the suitability of the new scale to be used with developmental samples. Finally, by comparing the performance of adolescents and young adults on the CRT and CRT‐long, we report the first investigation into the development of cognitive reflection. Copyright © 2015 John Wiley & Sons, Ltd.     

Test of memory malingering with children: The utility of Trial 1 and TOMMe10 as screeners of test validity

There is a growing body of research suggesting that the shorter versions of the Test of Memory Malingering (TOMM) may provide an accurate assessment of effort in children. During neuropsychological evaluations, some circumstances result in only one completed trial of the TOMM or partial completion of a trial. Research suggests that a cut-score of 40 or 41 on Trial1 is highly predictive of passing the TOMM overall. In the current study, 194 school-age children with academic and/or behavioral problems were used to compare the accuracy of TOMM1 and TOMMe10 (errors on the first 10 items of TOMM1) in predicting passing/failing of TOMM2. For the children in this sample, a score of < 40 items correct (≥ 10 errors) on TOMM1 was highly accurate in predicting a passing performance on the TOMM2 (sensitivity = .80, specificity = .91) with a Negative Predictive Value = .98 at the malingering base rate of 7% (TOMM2 failure in our sample). A score of 2 errors (8 items correct) on the TOMMe10 was slightly less sensitive than that of the TOMM1 (specificity = .96, sensitivity = .53) but with a similar Negative Predictive Value (.96). Consistent with the research from adult populations, TOMM1 and TOMMe10 appear to be quite accurate in predicting performance on the standard administration of the TOMM and may be useful screeners. However, compared to that found in adult samples, slight differences in suggested cutoffs for TOMM1 and TOMMe10 may be warranted for children.     

A statistical model of cleavage fracture in structural steels with power-law distribution of microcrack size

Based on the well-accepted physical understanding that cleavage fracture of structural steels is preceded by plastic deformation, a critical flaw in the Beremin model and its modifications is proposed, and a new statistical model with the power-law distribution of microcracks is obtained to describe the cumulative probability of cleavage fracture. A set of cleavage fracture toughness data of a nuclear pressure vessel steel is used to highlight the difference between the new model and the Beremin model.     

The International Test Commission Guidelines on the Security of Tests,Examinations, and Other Assessments

The amount and severity of security threats have increased considerably over the past two decades, calling into question the validity of assessments administered around the world. These threats have increased for a number of reasons, including the popular use of computerized and online technologies for test administration and the use of almost undetectable technologies for capturing test content and illegally sharing it instantly across borders and cultures. No assessment program, large or small, is immune to this potential damage. The International Test Commission has recognized the critical need for every organization with an important assessment program to be aware of these and be prepared to counter them. It was for this purpose that these guidelines were developed. Knowing the threats and the guidelines will lead to effective measures to protect the program and its assets, maintaining the value of the tests and assessments to the international community. The guidelines listed in this article provide recommendations on planning for better security, maintaining security during the development of tests and while they are administered, and responding well when a security breach occurs. Following these guidelines will create a significant protective barrier between those who willingly commit test fraud and the valuable assets a program has spent time and money to build.     

Psychometrics in Support of a Valid Assessment of Linguistic Minorities: Implications for the Test and Sampling Designs

In this study, we propose that the unique needs and characteristics of linguistic minorities should be considered throughout the test development process. Unlike most measurement invariance investigations in the assessment of linguistic minorities, which typically are conducted after test administration, we propose strategies that focus on the early stages of test development. Our approaches involve considering linguistic minorities in the selection of the test and sampling designs. We posit that joint consideration of these issues preemptively can strengthen the claims derived from tests used to assess linguistically diverse populations. This also will contribute to more psychometrically robust assessments, which can yield accurate and valid score-based inferences for linguistic minorities. To this end, we provide various examples and methodological approaches that can be used in the selection of the test and sampling designs that take these issues into consideration.     

Disentangling proactive and reactive aggression in children using self-report

The distinction between proactive and reactive functions of aggression is one of the most common divisions when investigating aggression among children and adolescents. To date, self-report is the least used measurement, despite existing literature supporting the view that the best informant regarding internal processes and motives are children themselves. The main aim of this study was to examine the construct and concurrent validity of a new self-report questionnaire, which aims to disentangle acts of reactive vs. proactive aggression that are most common within the daily lives of children. We examined the self-report measure among 578 children (313 girls, 265 boys, mean age 11 years, range 9–13 years). Most children (90% boys; 85% girls) reported at least one act of aggression over the last four weeks. Furthermore, the outcomes support the two-factor structure (reactive and proactive aggression) and the questionnaire showed good concurrent and discriminant validity with measures for emotional and social functioning. This study validates the use of the self-report instrument for reactive and proactive aggression and demonstrates that children can successfully distinguish between their own motives for reactive and proactive forms of aggressive behaviours.     

Construct validity and construct equivalence of work values across boys and girls

This study investigated the structural validity and construct comparability, across boys and girls, of a newly developed measure of work values. The total sample was split into two equal parts, and based on an item-level factor analysis of the first sub-sample, fit statistics provided support for a six-factor first-order structural model with the following constructs: Power, Money, Tidy environment, Teamwork, Work–Family Balance and Ability Utilization. This structure was confirmed in the second sub-sample, and strict measurement invariance for the measurement model was obtained in this sample. Additionally, the factor variances/covariances were invariant. Five non-invariant latent means were found, with girls scoring higher on the Balance and Personal Challenge values, and boys scoring higher on the Power, Money and Teamwork values.