On the sampling theory roundations of item response theory models

Item response theory (IT) models are now in common use for the analysis of dichotomous item responses. This paper examines the sampling theory foundations for statistical inference in these models. The discussion includes: some history on the stochastic subject versus the random sampling interpretations of the probability in IRT models; the relationship between three versions of maximum likelihood estimation for IRT models; estimating versus estimating -predictors; IRT models and loglinear models; the identifiability of IRT models; and the role of robustness and Bayesian statistics from the sampling theory perspective.A presidential address can serve many different functions. This one is a report of investigations I started at least ten years ago to understand what IRT was all about. It is a decidedly one-sided view, but I hope it stimulates controversy and further research. I have profited from discussions of this material with many people including: Brian Junker, Charles Lewis, Nicholas Longford, Robert Mislevy, Ivo Molenaar, Donald Rock, Donald Rubin, Lynne Steinberg, Martha Stocking, William Stout, Dorothy Thayer, David Thissen, Wim van der Linden, Howard Wainer, and Marilyn Wingersky. Of course, none of them is responsible for any errors or misstatements in this paper. The research was supported in part by the Cognitive Science Program, Office of Naval Research under Contract No. Nooo14-87-K-0730 and by the Program Statistics Research Project of Educational Testing Service.     

Probability and Evolution. Why the Probability Argument of Creationists is Wrong

Summary  Evolution is a time process. It proceeds in steps of definite length. The probability of each step is relatively high, so self organization of complex systems will be possible in finite time. Prerequisite for such a process is a selection rule, which certainly exists in evolution. Therefore, it would be wrong to calculate the probability of the formation of a complex system solely on the basis of the number of its components and as a momentary event.     

Expectation-based syntactic comprehension

This paper investigates the role of resource allocation as a source of processing difficulty in human sentence comprehension. The paper proposes a simple information-theoretic characterization of processing difficulty as the work incurred by resource reallocation during parallel, incremental, probabilistic disambiguation in sentence comprehension, and demonstrates its equivalence to the theory of Hale [Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. In Proceedings of NAACL (Vol. 2, pp. 159-166)], in which the difficulty of a word is proportional to its surprisal (its negative log-probability) in the context within which it appears. This proposal subsumes and clarifies findings that high-constraint contexts can facilitate lexical processing, and connects these findings to well-known models of parallel constraint-based comprehension. In addition, the theory leads to a number of specific predictions about the role of expectation in syntactic comprehension, including the reversal of locality-based difficulty patterns in syntactically constrained contexts, and conditions under which increased ambiguity facilitates processing. The paper examines a range of established results bearing on these predictions, and shows that they are largely consistent with the surprisal theory.     

Structural constraints in the perception of English stop-sonorant clusters

Native-language phonemes combined in a non-native way can be misperceived so as to conform to native phonotactics, e.g. English listeners are biased to hear syllable-initial [tr] rather than the illegal [tl] (Perception and Psychophysics 34 (1983) 338; Perception and Psychophysics 60 (1998) 941). What sort of linguistic knowledge causes phonotactic perceptual bias? Two classes of models were compared: unit models, which attribute bias to the listener's differing experience of each cluster (such as their different frequencies), and structure models, which use abstract phonological generalizations (such as a ban on [coronal][coronal] sequences). Listeners (N=16 in each experiment) judged synthetic 6 x 6 arrays of stop-sonorant clusters in which both consonants were ambiguous. The effect of the stop judgment on the log odds ratio of the sonorant judgment was assessed separately for each stimulus token to provide a stimulus-independent measure of bias. Experiment 1 compared perceptual bias against the onsets [bw] and [dl], which violate different structural constraints but are both of zero frequency. Experiment 2 compared bias against [dl] in CCV and VCCV contexts, to investigate the interaction of syllabification with segmentism and to rule out a compensation-for-coarticulation account of Experiment 1. Results of both experiments favor the structure models (supported by NSF).     

Molecular analyses of the principal components of response strength

Killeen and Hall (2001) showed that a common factor called strength underlies the key dependent variables of response probability, latency, and rate, and that overall response rate is a good predictor of strength. In a search for the mechanisms that underlie those correlations, this article shows that (a) the probability of responding on a trial is a two-state Markov process; (b) latency and rate of responding can be described in terms of the probability and period of stochastic machines called clocked Bernoulli modules, and (c) one such machine, the refractory Poisson process, provides a functional relation between the probability of observing a response during any epoch and the rate of responding. This relation is one of proportionality at low rates and curvilinearity at higher rates.     

Entailment with Near Surety of Scaled Assertions of High Conditional Probability

An assertion of high conditional probability or, more briefly, an HCP assertion is a statement of the type: The conditional probability of B given A is close to one. The goal of this paper is to construct logics of HCP assertions whose conclusions are highly likely to be correct rather than certain to be correct. Such logics would allow useful conclusions to be drawn when the premises are not strong enough to allow conclusions to be reached with certainty. This goal is achieved by taking Adams" (1966) logic, changing its intended application from conditionals to HCP assertions, and then weakening its criterion for entailment. According to the weakened entailment criterion, called the Criterion of Near Surety and which may be loosely interpreted as a Bayesian criterion, a conclusion is entailed if and only if nearly every model of the premises is a model of the conclusion. The resulting logic, called NSL, is nonmonotonic. Entailment in this logic, although not as strict as entailment in Adams" logic, is more strict than entailment in the propositional logic of material conditionals. Next, NSL was modified by requiring that each HCP assertion be scaled; this means that to each HCP assertion was associated a bound on the deviation from 1 of the conditional probability that is the subject of the assertion. Scaling of HCP assertions is useful for breaking entailment deadlocks. For example, it it is known that the conditional probabilities of C given A and of ¬ C given B are both close to one but the bound on the former"s deviation from 1 is much smaller than the latter"s, then it may be concluded that in all likelihood the conditional probability of C given A B is close to one. The resulting logic, called NSL-S, is also nonmonotonic. Despite great differences in their definitions of entailment, entailment in NSL is equivalent to Lehmann and Magidor"s rational closure and, disregarding minor differences concerning which premise sets are considered consistent, entailment in NSL-S is equivalent to entailment in Goldszmidt and Pearl"s System-Z ⁺. Bacchus, Grove, Halpern, and Koller proposed two methods of developing a predicate calculus based on the Criterion of Near Surety. In their random-structures method, which assumed a prior distribution similar to that of NSL, it appears possible to define an entailment relation equivalent to that of NSL. In their random-worlds method, which assumed a prior distribution dramatically different from that of NSL, it is known that the entailment relation is different from that of NSL.     

On the effects of signaling reinforcer probability and magnitude in delayed matching to sample

Two experiments examined whether postsample signals of reinforcer probability or magnitude affected the accuracy of delayed matching to sample in pigeons. On each trial, red or green choice responses that matched red or green stimuli seen shortly before a variable retention interval were reinforced with wheat access. In Experiment 1, the reinforcer probability was either 0.2 or 1.0 for both red and green responses. Reinforcer probability was signaled by line or cross symbols that appeared after the sample had been presented. In Experiment 2, all correct responses were reinforced, and the signaled reinforcer durations were 1.0 s and 4.5 s. Matching was more accurate when larger or more probable reinforcers were signaled, independently of retention interval duration. Because signals were presented postsample, the effects were not the result of differential attention to the sample.     

Speech and non-speech processing in people with specific language impairment: a behavioural and electrophysiological study

McArthur and Bishop (2004) found that people with specific language impairment (SLI) up to 14 years of age have poor behavioural frequency discrimination (FD) thresholds for 25-ms pure tones, while people with SLI upto 20 years of age have abnormal auditory N1--P2--N2 event-related potential (ERP) responses to the same tones. In the present study, we extended these findings to more complex non-speech and speech sounds by comparing younger (around 13 years) and older (around 17 years) teenagers with SLI and controls for their behavioural FD thresholds and N1-P2 ERPs to 25 and 250-ms pure tones, vowels, and non-harmonic complex tones. We found that a subgroup of people with SLI had abnormal responses to tones and vowels at the level of behaviour and the brain, and that poor processing was associated with the spectral complexity of auditory stimuli rather than their phonetic significance. We suggest that both the age of listeners and the sensitivity of psychoacoustic tasks to age-related changes in auditory skills may be crucial factors in studies of sound processing in SLI.     

Neighbourhood effects on error rates in speech production

Models of speech production differ on whether phonological neighbourhoods should affect processing, and on whether effects should be facilitatory or inhibitory. Inhibitory effects of large neighbourhoods have been argued to underlie apparent anti-frequency effects, whereby high-frequency default features are more prone to mispronunciation errors than low-frequency nondefault features. Data from the original SLIPs experiments that found apparent anti-frequency effects are analysed for neighbourhood effects. Effects are facilitatory: errors are significantly less likely for words with large numbers of neighbours that share the characteristic that is being primed for error ("friends"). Words in the neighbourhood that do not share the target characteristic ("enemies") have little effect on error rates. Neighbourhood effects do not underlie the apparent anti-frequency effects. Implications for models of speech production are discussed.