The case for comparability

We argue that all comparative expressions in natural language obey a principle that we call Comparability: if $x$ and $y$ are at least as $F$ as themselves, then either $x$ is at least as $F$ as $y$ or $y$ is at least as $F$ as $x$. This principle has been widely rejected among philosophers, especially by ethicists, and its falsity has been claimed to have important normative implications. We argue that Comparability is needed to explain the goodness of several patterns of inference that seem manifestly valid, that the purported failures of Comparability would have absurd consequences, and that the influential arguments against Comparability are less compelling than they may have initially seemed.     

Prior's puzzle generalized

Prior's puzzle is standardly taken to be the puzzle of why, given the assumption that that-clauses denote propositions, substitution of $⌜$the proposition that $P$$⌝$ for $⌜$that $P$$⌝$ within the complements of many propositional attitude verbs is invalid. I show that Prior's puzzle is much more general than is ordinarily supposed. There are two variants on the substitutional form of the puzzle—a quantificational variant and a pronominal variant—and all three forms of the puzzle arise in a wide range of verbal complements, rather than merely in the complements of propositional attitude verbs. The generalized puzzle shows that a range of proposed solutions to the original puzzle fail, or are radically incomplete, and also reveals the connections between Prior's puzzle and debates over the nature of semantic types and higher-order quantification. I go on to develop a novel, higher-order solution to the generalized form of the puzzle, and I argue that this higher-approach is superior to its first-order alternatives.     

The criterion-related validity of conscientiousness in personnel selection: A meta-analytic reality check

A key finding in personnel selection is the positive correlation between conscientiousness and job performance. Evidence predominantly stems from concurrent validation studies with incumbent samples but is readily generalized to predictive settings with job applicants. This is problematic because the extent to which faking and changes in personality affect the measurement likely vary across samples and study designs. Therefore, we meta-analytically investigated the relation between conscientiousness and job performance, examining the moderating effects of sample type (incumbent vs. applicant) and validation design (concurrent vs. predictive). The overall correlation of conscientiousness and job performance was in line with previous meta-analyses ($\overline{r}=.17,k=102,n=23,305$). In our analyses, the correlation did not differ across validation designs (concurrent: $\overline{r}=.18,k=78,n=19,132$; predictive: $\overline{r}=.15,k=24,n=4173$), sample types (incumbents: $\overline{r}=.18,k=92,n=20,808$; applicants: $\overline{r}=.14,k=10,n=2497$), or their interaction. Critically, however, our review revealed that only a small minority of studies (~12%) were conducted with real applicants in predictive designs. Thus, barely a fraction of research is conducted under realistic conditions. Therefore, it remains an open question if self-report measures of conscientiousness retain their predictive validity in applied settings that entail faked responses. We conclude with a call for more multivariate research on the validity of selection procedures in predictive settings with actual applicants.     

Assessment of generalised Bayesian structural equation models for continuous and binary data

The paper proposes a novel model assessment paradigm aiming to address shortcoming of posterior predictive $p$-values, which provide the default metric of fit for Bayesian structural equation modelling (BSEM). The model framework presented in the paper focuses on the approximate zero approach (Psychological Methods, 17 , 2012, 313), which involves formulating certain parameters (such as factor loadings) to be approximately zero through the use of informative priors, instead of explicitly setting them to zero. The introduced model assessment procedure monitors the out-of-sample predictive performance of the fitted model, and together with a list of guidelines we provide, one can investigate whether the hypothesised model is supported by the data. We incorporate scoring rules and cross-validation to supplement existing model assessment metrics for BSEM. The proposed tools can be applied to models for both continuous and binary data. The modelling of categorical and non-normally distributed continuous data is facilitated with the introduction of an item-individual random effect. We study the performance of the proposed methodology via simulation experiments as well as real data on the ‘Big-5’ personality scale and the Fagerstrom test for nicotine dependence.     

Nothing But Stereotypes? Negligible Sex Differences Across Creativity Measures in Science,Arts, and Sports Adolescent High Achievers

Previous research has focused on understanding when, why, and how sex differences in creativity occur, as results vary across samples, measures, and methodologies. In the current study we investigated sex differences in creativity among 984 high achieving adolescents in three expertise areas: Sciences, Arts, and Sports. Eight creativity indicators were analyzed: Alternative uses task (AUT) fluency; creative self-efficacy (CSE); intraindividual strengths (difference between CSE and AUT Fluency); five self-reported creativity scales: Self/everyday, scholarly, performance, mechanical/scientific, artistic. The results showed negligible sex differences ( $n_{\mathrm{p}}^2$ = .01), with females performing better in AUT Fluency and males self-rating their CSE higher. No sex differences were found in self/everyday, scholarly and performance creativity. Males self-rated their mechanical/scientific creativity ( $n_{\mathrm{p}}^2$ = .06) higher than females; while females self-rated their artistic creativity ( $n_{\mathrm{p}}^2$ = .02) higher in comparison to males. Our results extend the existing literature by finding negligible sex differences in adolescent expert groups. However, some stereotypical differences emerged, for example, females with Sciences expertise rated their mechanical/scientific creativity lower than males with and even without Sciences expertise. Results call for further investigation into the links between sex differences, expertise, and specific creativity domains.     

Rectifying the Mischaracterization of Logic by Mental Model Theorists

Khemlani et al. (2018) mischaracterize logic in the course of seeking to show that mental model theory (MMT) can accommodate a form of inference (, let us label it) they find in a high percentage of their subjects. We reveal their mischaracterization and, in so doing, lay a landscape for future modeling by cognitive scientists who may wonder whether human reasoning is consistent with, or perhaps even capturable by, reasoning in a logic or family thereof. Along the way, we note that the properties touted by Khemlani et al. as innovative aspects of MMT-based modeling (e.g., nonmonotonicity) have for decades been, in logic, acknowledged and rigorously specified by families of (implemented) logics. Khemlani et al. (2018) further declare that is “invalid in any modal logic.” We demonstrate this to be false by our introduction (Appendix A) of a new propositional modal logic (within a family of such logics) in which is provably valid, and by the implementation of this logic. A second appendix, B, partially answers the two-part question, “What is a formal logic, and what is it for one to capture empirical phenomena?”     

Notes on attribution functions

Let be the knowledge space derived from an attribution function σ on Q. Under an assumption for σ, this paper gives some necessary and sufficient conditions such that is discriminative. It also discusses the resolubility of σ when Q is an infinite set. More precisely, this paper proves that σ is not resoluble if Q is uncountable, and gives a necessary and sufficient condition such that σ is resoluble when is -well-graded. By way of applications of these results, discriminativeness and resolubility are discussed around the merge of skill multimaps and the meshing of the delineated knowledge spaces.     

A new goodness-of-fit measure for probit models: Surrogate R2

Probit models are used extensively for inferential purposes in the social sciences as discrete data are prevalent in a vast body of social studies. Among many accompanying model inference problems, a critical question remains unsettled: how to develop a goodness-of-fit measure that resembles the ordinary least square (OLS) R² used for linear models. Such a measure has long been sought to achieve ‘comparability’ of different empirical models across multiple samples addressing similar social questions. To this end, we propose a novel R² measure for probit models using the notion of surrogacy – simulating a continuous variable $S$ as a surrogate of the original discrete response (Liu & Zhang, Journal of the American Statistical Association, 113, 845 and 2018). The proposed R² is the proportion of the variance of the surrogate response explained by explanatory variables through a linear model, and we call it a surrogate R². This paper shows both theoretically and numerically that the surrogate R² approximates the OLS R² based on the latent continuous variable, preserves the interpretation of explained variation, and maintains monotonicity between nested models. As no other pseudo R², McKelvey and Zavoina's and McFadden's included, can meet all the three criteria simultaneously, our measure fills this crucial void in probit model inference.     

‘MusiMath’ and ‘Academic Music’ – Two music‐based intervention programs for fractions learning in fourth grade students

Music and mathematics require abstract thinking and using symbolic notations. Controversy exists regarding transfer from musical training to math achievements. The current study examined the effect of two integrated intervention programs representing holistic versus acoustic approaches, on fraction knowledge. Three classes of fourth graders attended 12 lessons on fractions: One class attended the ‘MusiMath’ holistic program (n = 30) focusing on rhythm within the melody. Another class attended the ‘Academic Music’ acoustic program (Courey et al., Educ Stud Math 81:251, 2012) (n = 25) which uses rhythm only. The third class received regular fraction lessons (comparison group, n = 22). Students in both music programs learned to write musical notes and perform rhythmic patterns through clapping and drumming as part of their fraction lessons. They worked toward adding musical notes to produce a number (fraction), and created addition/subtraction problems with musical notes. The music programs used a 4/4 time signature with whole, half, quarter and eighth notes. In the math lessons, the students learned the analogy between musical durations and fractions, but also practiced fractions other than . Music and math were assessed before, immediately following, and 3‐ and 6‐months post‐intervention. Pre‐ to post‐intervention analyses indicated that only the ‘MusiMath’ group showed greater transfer to intervention‐trained and untrained fractions than the comparison group. The ‘Academic Music’ group showed a trend on trained fractions. Although both music groups outperformed the comparison group 3‐ and 6‐months post‐intervention on trained fractions, only the ‘MusiMath’ group demonstrated greater gains in untrained fractions. Gains were more evident in trained than in untrained fractions. A video abstract of this article can be viewed at https://youtu.be/uJ_KWWDO624     

Limited‐information goodness‐of‐fit testing of diagnostic classification item response models

Despite the growing popularity of diagnostic classification models (e.g., Rupp et al., 2010, Diagnostic measurement: theory, methods, and applications, Guilford Press, New York, NY) in educational and psychological measurement, methods for testing their absolute goodness of fit to real data remain relatively underdeveloped. For tests of reasonable length and for realistic sample size, full‐information test statistics such as Pearson's X² and the likelihood ratio statistic G² suffer from sparseness in the underlying contingency table from which they are computed. Recently, limited‐information fit statistics such as Maydeu‐Olivares and Joe's (2006, Psychometrika, 71, 713) M₂ have been found to be quite useful in testing the overall goodness of fit of item response theory models. In this study, we applied Maydeu‐Olivares and Joe's (2006, Psychometrika, 71, 713) M₂ statistic to diagnostic classification models. Through a series of simulation studies, we found that M₂ is well calibrated across a wide range of diagnostic model structures and was sensitive to certain misspecifications of the item model (e.g., fitting disjunctive models to data generated according to a conjunctive model), errors in the Q‐matrix (adding or omitting paths, omitting a latent variable), and violations of local item independence due to unmodelled testlet effects. On the other hand, M₂ was largely insensitive to misspecifications in the distribution of higher‐order latent dimensions and to the specification of an extraneous attribute. To complement the analyses of the overall model goodness of fit using M₂, we investigated the utility of the Chen and Thissen (1997, J. Educ. Behav. Stat., 22, 265) local dependence statistic $X_{\mathrm{LD}}^2$ for characterizing sources of misfit, an important aspect of model appraisal often overlooked in favour of overall statements. The $X_{\mathrm{LD}}^2$ statistic was found to be slightly conservative (with Type I error rates consistently below the nominal level) but still useful in pinpointing the sources of misfit. Patterns of local dependence arising due to specific model misspecifications are illustrated. Finally, we used the M₂ and $X_{\mathrm{LD}}^2$ statistics to evaluate a diagnostic model fit to data from the Trends in Mathematics and Science Study, drawing upon analyses previously conducted by Lee et al., (2011, IJT, 11, 144).     

Can we disregard the whole model? Omnibus non-inferiority testing for R2 in multi-variable linear regression and in ANOVA

Determining a lack of association between an outcome variable and a number of different explanatory variables is frequently necessary in order to disregard a proposed model (i.e., to confirm the lack of a meaningful association between an outcome and predictors). Despite this, the literature rarely offers information about, or technical recommendations concerning, the appropriate statistical methodology to be used to accomplish this task. This paper introduces non-inferiority tests for ANOVA and linear regression analyses, which correspond to the standard widely used F test for and R², respectively. A simulation study is conducted to examine the Type I error rates and statistical power of the tests, and a comparison is made with an alternative Bayesian testing approach. The results indicate that the proposed non-inferiority test is a potentially useful tool for ‘testing the null’.     

IN PRAISE OF A LOGIC OF DEFINITIONS THAT TOLERATES Ω‐INCONSISTENCY

I argue that a general logic of definitions must tolerate ω‐inconsistency. I present a semantical scheme, , under which some definitions imply ω‐inconsistent sets of sentences. I draw attention to attractive features of this scheme, and I argue that yields the minimal general logic of definitions. I conclude that any acceptable general logic should permit definitions that generate ω‐inconsistency. This conclusion gains support from the application of to the theory of truth.     

The Latent Threat of Community Violence: Indirect Exposure to Local Homicides and Adolescents’ Mental Health in Colombia

This study examines the relation between adolescents’ indirect exposure to local homicides and mental health disorders and post‐traumatic stress disorder (PTSD) symptoms. We employ a sample of 300 adolescents ( representative for Bogotá, Colombia, and geocoded data on violent crimes recorded by the national police. Findings show that one SD increment in local homicides is associated with increments by 0.17 SD in the mental health disorder index and a 0.14 SD increase in the PTSD score index, even after accounting for adolescents’ direct exposure to violence. The estimated effect for PTSD was larger for adolescents’ who were directly exposed to violence and for those living in multidimensionally poor households, whereas no detectable effects were found for adolescents who perceived their residential neighborhood as relatively safe.     

Estimation of the predictive power of the model in mixed‐effects meta‐regression: A simulation study

Several methods are available to estimate the total and residual amount of heterogeneity in meta‐analysis, leading to different alternatives when estimating the predictive power in mixed‐effects meta‐regression models using the formula proposed by Raudenbush (1994, 2009). In this paper, a simulation study was conducted to compare the performance of seven estimators of these parameters under various realistic scenarios in psychology and related fields. Our results suggest that the number of studies (k) exerts the most important influence on the accuracy of the results, and that precise estimates of the heterogeneity variances and the model predictive power can only be expected with at least 20 and 40 studies, respectively. Increases in the average within‐study sample size () also improved the results for all estimators. Some differences among the accuracy of the estimators were observed, especially under adverse (small k and ) conditions, while the results for the different methods tended to convergence for more optimal scenarios.     

Modelling monotonic effects of ordinal predictors in Bayesian regression models

Ordinal predictors are commonly used in regression models. They are often incorrectly treated as either nominal or metric, thus under- or overestimating the information contained. Such practices may lead to worse inference and predictions compared to methods which are specifically designed for this purpose. We propose a new method for modelling ordinal predictors that applies in situations in which it is reasonable to assume their effects to be monotonic. The parameterization of such monotonic effects is realized in terms of a scale parameter b representing the direction and size of the effect and a simplex parameter modelling the normalized differences between categories. This ensures that predictions increase or decrease monotonically, while changes between adjacent categories may vary across categories. This formulation generalizes to interaction terms as well as multilevel structures. Monotonic effects may be applied not only to ordinal predictors, but also to other discrete variables for which a monotonic relationship is plausible. In simulation studies we show that the model is well calibrated and, if there is monotonicity present, exhibits predictive performance similar to or even better than other approaches designed to handle ordinal predictors. Using Stan, we developed a Bayesian estimation method for monotonic effects which allows us to incorporate prior information and to check the assumption of monotonicity. We have implemented this method in the R package brms, so that fitting monotonic effects in a fully Bayesian framework is now straightforward.     

Individual Differences in Risky Decision Making: A Meta‐analysis of Sensation Seeking and Impulsivity with the Balloon Analogue Risk Task

To represent the state‐of‐the‐art in an effort to understand the relation between personality and risk taking, we selected a popular decision task with characteristics that parallel risk taking in the real world and two personality traits commonly believed to influence risk taking. A meta‐analysis is presented based on 22 studies of the Balloon Analogue Risk Task from which correlations with sensation seeking and impulsivity assessments could be obtained. Results calculated on a total of 2120 participants showed that effect size for the relation of sensation seeking with risk taking was in the small–moderate range ( = .14), whereas the effect size for impulsivity was just around the small effect size threshold ( = .10). Although we considered participants' demographics as moderators, we found only significantly larger effect sizes for the older adolescents and young adults compared with other ages. The findings of the present review supported the view that inconsistencies in personality–risk research were mostly due to random fluctuations of specific effect sizes, rather than to lack of theoretical ties or to measurement unreliability. It is also concluded that studies aimed at relating individual differences in personality to performance in experimental decision tasks need an appropriate sample size to achieve the power to produce significant results. Copyright © 2013 John Wiley & Sons, Ltd.