Using response times to detect aberrant responses in computerized adaptive testing

A lognormal model for response times is used to check response times for aberrances in examinee behavior on computerized adaptive tests. Both classical procedures and Bayesian posterior predictive checks are presented. For a fixed examinee, responses and response times are independent; checks based on response times offer thus information independent of the results of checks on response patterns. Empirical examples of the use of classical and Bayesian checks for detecting two different types of aberrances in response times are presented. The detection rates for the Bayesian checks outperformed those for the classical checks, but at the cost of higher false-alarm rates. A guideline for the choice between the two types of checks is offered.This study received funding from the Law School Admission Council (LSAC). The opinions and conclusions contained in this paper are those of the authors and do not necessarily reflect the policy and position of LSAC. The authors are most indebted to Wim M. M. Tielen for his computational assistance and to the US Defense Manpower Data Center for the permission to use the ASVAB data set in the empirical examples.     

Velocity dependence of the interocular transfer of dynamic motion aftereffects

It is well established that motion aftereffects (MAEs) can show interocular transfer (IOT); that is, motion adaptation in one eye can give a MAE in the other eye. Different quantification methods and different test stimuli have been shown to give different IOT magnitudes, varying from no to almost full IOT. In this study, we examine to what extent IOT of the dynamic MAE (dMAE), that is the MAE seen with a dynamic noise test pattern, varies with velocity of the adaptation stimulus. We measured strength of dMAE by a nulling method. The aftereffect induced by adaptation to a moving random-pixel array was compensated (nulled), during a brief dynamic test period, by the same kind of motion stimulus of variable luminance signal-to-noise ratio (LSNR). The LSNR nulling value was determined in a Quest-staircase procedure. We found that velocity has a strong effect on the magnitude of IOT for the dMAE. For increasing speeds from 1.5 deg s(-1) to 24 deg s(-1) average IOT values increased about linearly from 18% to 63% or from 32% to 83%, depending on IOT definition. The finding that dMAEs transfer to an increasing extent as speed increases, suggests that binocular cells play a more dominant role at higher speeds.     

Estimating the mean effect size in meta-analysis: Bias,precision, and mean squared error of different weighting methods

Although use of the standardized mean difference in meta-analysis is appealing for several reasons, there are some drawbacks. In this article, we focus on the following problem: that a precision-weighted mean of the observed effect sizes results in a biased estimate of the mean standardized mean difference. This bias is due to the fact that the weight given to an observed effect size depends on this observed effect size. In order to eliminate the bias, Hedges and Olkin (1985) proposed using the mean effect size estimate to calculate the weights. In the article, we propose a third alternative for calculating the weights: using empirical Bayes estimates of the effect sizes. In a simulation study, these three approaches are compared. The mean squared error (MSE) is used as the criterion by which to evaluate the resulting estimates of the mean effect size. For a meta-analytic dataset with a small number of studies, theMSE is usually smallest when the ordinary procedure is used, whereas for a moderate or large number of studies, the procedures yielding the best results are the empirical Bayes procedure and the procedure of Hedges and Olkin, respectively.     

Visual perspective taking for avatars in a Simon task

In modern digital applications, users often interact with virtual representations of themselves or others, called avatars. We examined how these avatars and their perspectives influence stimulus–response compatibility in a Simon task. Participants responded to light/dark blue stimuli with left/right key presses in the presence of a task-irrelevant avatar. Changes in stimulus–response compatibility were used to quantify changes in the mental representation of the task and perspective taking toward this avatar. Experiments 1 and 2 showed that perspective taking for an avatar occurred in orthogonal stimulus–response mappings, causing a compatibility effect from the avatar’s point of view. In the following two experiments we introduced a larger variety of angular disparities between the participant and avatar. In Experiment 3, the Simon effect with lateralized stimulus positions remained largely unaffected by the avatar, pointing toward an absence of perspective taking. In Experiment 4, after avatar hand movements were added in order to strengthen the participants’ sense of agency over the avatar, a spatial compatibility effect from the avatar’s perspective was observed again, and hints of the selective use of perspective taking on a trial-by-trial basis were found. Overall, the results indicate that users can incorporate the perspective of an avatar into their mental representation of a situation, even when this perspective is unnecessary to complete a task, but that certain contextual requirements have to be met.     

Knowledge level learning and the power law: A soar model of skill acquisition in scheduling

Zusammenfassung. Das hier vorgestellte Modell lernt graduell, Planungsaufgaben aus der Klasse der Maschinenbelegungsprobleme (job-shop-scheduling problems) zu lösen. Mit Hilfe des Chunking-Mechanismus von Soar wird episodisches Wissen über die Belegungsreihenfolge von Aufträgen auf Maschinen memoriert. Bei der Entwicklung des Modells wurden zahlreiche qualitative (z. B. Transfereffekte) und quantitative Befunde (z. B. Bearbeitungszeiten) aus einer früheren empirischen Untersuchung berücksichtigt. In einer Validierungsstudie wurden dieselben Aufgaben von 14 Probanden und dem Modell bearbeitet. Die Passung von Simulationsdaten und empirischen Ergebnissen fiel insgesamt gut aus. Allerdings löst das Modell die Aufgaben schneller und zeigt auch einen etwas besseren Lernverlauf als die Probanden. Das Modell liefert eine Erklärung für das Rauschen, das typischerweise bei Bearbeitungszeiten zu beobachten ist: es handelt sich um erworbenes Wissen, das mehr oder weniger gut und auch unterschiedlich häufig auf neue Situationen übertragen wird. Der Lernverlauf der Probanden entspricht nur für aggregierte Daten einer Potenzfunktion (power law). Der vorgestellte Mechanismus zeigt, wie ein symbolisches Modell der Informationsverarbeitung graduelle Verhaltensänderungen generiert und wie der offensichtliche Erwerb allgemeiner Prozeduren ohne explizites Lernen von deklarativen Regeln erfolgen kann. Es wird nahegelegt, daß es sich hier um die Modellierung einer Form impliziten Lernens handelt. Summary. The model presented here gradually learns how to perform a job-shop scheduling task. It uses Soar's chunking mechanism to acquire episodic memories about the order to schedule jobs. The model was based on many qualitative (e.g., transfer effects) and quantitative (e.g., solution time) regularities found in previously collected data. The model was tested with new data where scheduling tasks were given to the model and to 14 subjects. The model generally fit these data with the restrictions that the model performs the task (in simulated time) faster than the subjects, and its performance improves somewhat more quickly than the subjects' performance. The model provides an explanation of the noise typically found in problem solving times - it is the result of learning actual pieces of knowledge that transfer more or less to new situations but rarely by an average amount. Only when the data are averaged (i.e., over subjects) does the smooth power law appear. This mechanism demonstrates how symbolic models can exhibit a gradual change in behavior and how the apparent acquisition of general procedures can be performed without resorting to explicit declarative rule generation. We suggest that this may represent a type of implicit learning.     

The puzzle of language evolution

Linguistics must again concentrate on the evolutionary nature of language, so that language models are more realistic with respect to human natural languages and have a greater explanatory force. Multi-agent systems are proposed as a possible route to develop such evolutionary models and an example is given of a concrete experiment in the origins and evolution of word-meaning based on a multi-agent approach.     

The human element in car following models

This commentary distinguishes between the engineering-inspired formalisms used by Brackstone, M., and McDonald, M. (Transportation Research Part F (1999)) and the psychological knowledge of, and assumptions about human behaviour when car following. It outlines a preliminary mathematical model, based on psychological knowledge about car following behaviour by human drivers, which could be applied in traffic engineering, as a means of bringing these two ways of studying car following closer together.     

Moral principles as moral dispositions

What are moral principles? In particular, what are moral principles of the sort that (if they exist) ground moral obligations or—at the very least—particular moral truths? I argue that we can fruitfully conceive of such principles as real, irreducibly dispositional properties of individual persons (agents and patients) that are responsible for and thereby explain the moral properties of (e.g.) agents and actions. Such moral dispositions (or moral powers) are apt to be the metaphysical grounds of moral obligations and of particular truths about what is morally permissible, impermissible, etc. Moreover, they can do other things that moral principles are supposed to do: explain the phenomena “falling within their scope,” support counterfactuals, and ground moral necessities, “necessary connections” between obligating reasons and obligations. And they are apt to be the truthmakers for moral laws, or “lawlike” moral generalizations.