Practical Applications of Generalizability Theory for Designing,Evaluating, and Improving Psychological Assessments

In this article, we illustrate how generalizability theory (G-theory) can extend traditional assessment methods for designing, improving, and evaluating results from both objectively and subjectively scored measures of individual differences. Our illustrations include quantification of multiple sources of measurement error, derivation of unique indexes of consistency for norm- and criterion-referenced interpretations of scores, estimation of score consistency when changing a measurement procedure, and disattenuation of correlation coefficients for measurement error. We also expand G-theory analyses beyond the item level to include parcels and split measures and highlight linkages among G-theory, classical test theory, and structural equation modeling. Computer code and sample data are provided in online supplements to help readers apply the demonstrated techniques to their own assessments.     

Reasoning from an incompatibility: False dilemma fallacies and content effects

In the present studies, we investigated inferences from an incompatibility statement. Starting with two propositions that cannot be true at the same time, these inferences consist of deducing the falsity of one from the truth of the other or deducing the truth of one from the falsity of the other. Inferences of this latter form are relevant to human reasoning since they are the formal equivalent of a discourse manipulation called the false dilemma fallacy, often used in politics and advertising in order to force a choice between two selected options. Based on research on content-related variability in conditional reasoning, we predicted that content would have an impact on how reasoners treat incompatibility inferences. Like conditional inferences, they present two invalid forms for which the logical response is one of uncertainty. We predicted that participants would endorse a smaller proportion of the invalid incompatibility inferences when more counterexamples are available. In Study 1, we found the predicted pattern using causal premises translated into incompatibility statements with many and few counterexamples. In Study 2A, we replicated the content effects found in Study 1, but with premises for which the incompatibility statement is a non-causal relation between classes. These results suggest that the tendency to fall into the false dilemma fallacy is modulated by the background knowledge of the reasoner. They also provide additional evidence on the link between semantic information retrieval and deduction.     

Introduction to Bayesian Inference for Psychology

We introduce the fundamental tenets of Bayesian inference, which derive from two basic laws of probability theory. We cover the interpretation of probabilities, discrete and continuous versions of Bayes’ rule, parameter estimation, and model comparison. Using seven worked examples, we illustrate these principles and set up some of the technical background for the rest of this special issue of Psychonomic Bulletin & Review. Supplemental material is available via https://osf.io/wskex/.     

Be it ever so humble: Proposing a dual-dimension account and measurement of humility

What does it mean to be humble? We argue that humility is an epistemically and ethically aligned state of awareness – the experience of ourselves as a small part of a larger universe and as one among a host of other morally relevant beings. So conceived, humility can be operationalized and measured along the dual dimensions of low self-focus and high other-focus and is distinct from other related constructs (e.g., modesty and open-mindedness). We discuss our newly developed scale (Study 1 and 2), and provide preliminary validation using self-report (Study 3) and behavioral measures (Study 4), showing that humility is related to people’s general ethical orientation (e.g., empathy, universalism/benevolence, and civic responsibility), their well-being (e.g., sense of autonomy, life-purpose, and secure attachment), mature religious beliefs/practices, and reactions to disagreement – specifically, people high in humility sat closer and less angled away from their conversation partner with whom they disagreed. Together, this provides support for our new Dual-Dimension Humility Scale.     

Predication

In Latent Semantic Analysis (LSA) the meaning of a word is represented as a vector in a high-dimensional semantic space. Different meanings of a word or different senses of a word are not distinguished. Instead, word senses are appropriately modified as the word is used in different contexts. In N-VP sentences, the precise meaning of the verb phrase depends on the noun it is combined with. An algorithm is described to adjust the meaning of a predicate as it is applied to different arguments. In forming a sentence meaning, not all features of a predicate are combined with the features of the argument, but only those that are appropriate to the argument. Hence, a different "sense" of a predicate emerges every time it is used in a different context. This predication algorithm is explored in the context of four different semantic problems: metaphor interpretation, causal inferences, similarity judgments, and homonym disambiguation.