Response error and processing biases in confidence judgment

Previous studies showed that random error can explain overconfidence effects typically observed in the literature. One of these studies concluded that, after accounting for random error effects in the data, there is little support for cognitive‐processing biases in confidence elicitation. In this paper, we investigate more closely the random error explanation for overconfidence. We generated data from four models of confidence and then estimated the magnitude of random error in the data. Our results show that, in addition to the true magnitude of random error specified in the simulations, the error estimates are influenced by important cognitive‐processing biases in the confidence elicitation process. We found that random error in the response process can account for the degree of overconfidence found in calibration studies, even when that overconfidence is actually caused by other factors. Thus, the error models say little about whether cognitive biases are present in the confidence elicitation process. Copyright © 2008 John Wiley & Sons, Ltd.     

Hierarchical models of simple mechanisms underlying confidence in decision making

Choice confidence is a central measure in psychological decision research, often being reported on a probabilistic scale. Simple mechanisms that describe the psychological processes underlying choice confidence, including those based on error and confirmation biases, have typically received support via fits to data averaged over subjects. While averaged data ease model development, they can also destroy important aspects of the confidence data distribution. In this paper, we develop a hierarchical model of raw confidence judgments using the beta distribution, and we implement two simple confidence mechanisms within it. We use Bayesian methods to fit the hierarchical model to data from a two-alternative confidence experiment, and we use a variety of Bayesian tools to diagnose shortcomings of the simple mechanisms that are overlooked when applied to averaged data. Bugs code for estimating the models is also supplied.     

Score-Based Tests of Differential Item Functioning via Pairwise Maximum Likelihood Estimation

Measurement invariance is a fundamental assumption in item response theory models, where the relationship between a latent construct (ability) and observed item responses is of interest. Violation of this assumption would render the scale misinterpreted or cause systematic bias against certain groups of persons. While a number of methods have been proposed to detect measurement invariance violations, they typically require advance definition of problematic item parameters and respondent grouping information. However, these pieces of information are typically unknown in practice. As an alternative, this paper focuses on a family of recently proposed tests based on stochastic processes of casewise derivatives of the likelihood function (i.e., scores). These score-based tests only require estimation of the null model (when measurement invariance is assumed to hold), and they have been previously applied in factor-analytic, continuous data contexts as well as in models of the Rasch family. In this paper, we aim to extend these tests to two-parameter item response models, with strong emphasis on pairwise maximum likelihood. The tests’ theoretical background and implementation are detailed, and the tests’ abilities to identify problematic item parameters are studied via simulation. An empirical example illustrating the tests’ use in practice is also provided.     

high-resolution electron microscopy at a (113) symmetric Thermally activated step motion observed by tilt grain-boundary in aluminium

Grain-boundary migration is demonstrated to proceed by lateral propagation of a small step in a (113), [110] symmetric Al tilt grain-boundary. In-situ high-resolution (transmission) electron microscopy (HREM) at 523K allowed the study of atomic-scale detail at video rates during the migration process. The grain-boundary translational states on both sides of the step are identical, which leads to a step dislocation. This defect can move laterally by a combination of climb and glide. Dynamic HREM images indicate considerable atomic agitation within the dislocation core. A detailed temporal analysis of the step movements shows small random displacements of the dislocation core.     

Comparing functional analysis and paired-choice assessment results in classroom settings

The results of a functional analysis of problem behavior and a paired-choice assessment were compared to determine whether the same social reinforcers were identified for problem behavior and an appropriate response (time allocation). The two assessments were conducted in classroom settings with 4 adolescents with mental retardation who engaged in severe problem behavior. Each student's classroom teacher served as the therapist for all phases of assessment. The two assessment procedures identified the same social reinforcers for problem and appropriate behavior for 3 of 4 participants.     

Testing for Measurement Invariance with Respect to an Ordinal Variable

Researchers are often interested in testing for measurement invariance with respect to an ordinal auxiliary variable such as age group, income class, or school grade. In a factor-analytic context, these tests are traditionally carried out via a likelihood ratio test statistic comparing a model where parameters differ across groups to a model where parameters are equal across groups. This test neglects the fact that the auxiliary variable is ordinal, and it is also known to be overly sensitive at large sample sizes. In this paper, we propose test statistics that explicitly account for the ordinality of the auxiliary variable, resulting in higher power against “monotonic” violations of measurement invariance and lower power against “non-monotonic” ones. The statistics are derived from a family of tests based on stochastic processes that have recently received attention in the psychometric literature. The statistics are illustrated via an application involving real data, and their performance is studied via simulation.     

Bayesian Comparison of Latent Variable Models: Conditional Versus Marginal Likelihoods

Psychometrika - Typical Bayesian methods for models with latent variables (or random effects) involve directly sampling the latent variables along with the model parameters. In high-level software...