An implicit enumeration method for an exact test of weighted kappa

The kappa coefficient is one of the most widely used measures for evaluating the agreement between two raters asked to assign N objects to one of K nominal categories. Weighted versions of kappa enable partial credit to be awarded for near agreement, most notably in the case of ordinal categories. An exact significance test for weighted kappa can be conducted by enumerating all rater agreement tables with the same fixed marginal frequencies as the observed table, and accumulating the probabilities for all tables that produce a weighted kappa index that is greater than or equal to the observed measure. Unfortunately, complete enumeration of all tables is computationally unwieldy for modest values of N and K. We present an implicit enumeration algorithm for conducting an exact test of weighted kappa, which can be applied to tables of non‐trivial size. The algorithm is particularly efficient for ‘good’ to ‘excellent’ values of weighted kappa that typically have very small p‐values. Therefore, our method is beneficial for situations where resampling tests are of limited value because the number of trials needed to estimate the p‐value tends to be large.     

An exact and optimal standardized person test for assessing consistency with the rasch model

The Rasch model predicts that an individual's ability level is invariant over subtests of the total test, and thus, all subtests measure the same latent trait. A person test of this invariance hypothesis is discussed that is uniformly most powerful and standardized in the sense that the conditional distribution of the test statistic, given a particular level of ability, does not depend on the absolute value of the examinee's ability parameter. The test can be routinely performed by applying a computer program designed by and obtainable from the author. Finally, a suboptimal test is derived that is extremely easy to use, and an overall group test of the invariance hypothesis discussed. All tests considered do not rely on asymptotic approximations; hence, they may be applied when the test is of only moderate length and the group of examinees is small.     

Comparisons of power for some exact multinomial significance tests

Two small-sample tests (proposed by Tate and Clelland and by Chapanis respectively) of hypotheses about the parameters of the multinomial distribution, where

$$f(x|p) = n!\prod\limits_{i = 1}^k {\frac{{p_i^{x_i } }}{{x_i !}}} $$     

Processing of repeated letters in search and matching tasks

Several published experiments have used a variant of the matching or “same”-“different” paradigm in which the subject indicates whether or not all of the items in a visual array are the same. Contrary to the results of most paradigms in which subjects process multiple items, reaction time (RT) in this matching task is independent of the number of items in the display (N). One possible explanation of this independence holds that subjects respond to some overall property of the display, such as its general symmetry, and not to the individual items. I tested this hypothesis with a visual search task in which subjects searched a circular display of N=2, 4, or 6 letters for a specified target letter under three stimulus conditions. In the varied condition, the nontarget letters were all different from each other. In the repeated condition, the nontargets were repetitions of the same letter, but a single discrepant letter was always present in the display. The uniform condition also used repeated nontargets, but the negative (target-absent) displays contained N repetitions of a single nontarget item. I found that RT increased at a rate of 27 msec/item for the varied condition, but at only 13 and 8 msec/item in the repeated and uniform conditions, respectively. The slopes in the latter conditions were significantly lower than the slope for the varied condition, and they also differed significantly from each other. Hence, there was some advantage attributable to properties of the overall patterns associated with positive and negative responses, suggesting that at least some responses were based on these properties. There was a much larger advantage, however, attributable to the use of repeated nontargets, regardless of whether the overall patterns differed. This latter observation implies that the flat RT functions observed in matching studies must be interpreted primarily in terms of responses to individual items rather than to the pattern as a whole.     

A simplified accurate algorithm for the fisher-yates exact test

An algorithm, using a short cut due to Feldman and Klingler, for the Fisher-Yates exact test is presented: the algorithm is quick, simple and accurate.I am grateful to the Editor and two reviewers for a number of constructive and helpful suggestions.     

An on-line integrator for alpha quantification

A system for the on-line graphic quantification of alpha frequencies (or other frequency bands extracted from EEG signals) is described. Alpha-frequency components (8 to 13 Hz) are actively filtered from complex EEG waveforms and presented to a laboratory-developed solid state integrator. The high-level output of the integrator, which is compatible with all popular EEG and polygraph recorders, is in the form of an ascending analog voltage ramp, reset at a predetermined level, to provide a rapid quantitative determination of alpha activity and variability. The design of the integrator permits an alternate mode of operation where time-synchronized resets, in lieu of voltage-dependent resets, produce variable height voltage ramps, whose amplitude is a direct function of the accumulated alpha activity per unit time. This method of preprocessing and signal conditioning the alpha components of the EEG waveform provides a simple analog registry of on-line real-time correlation of alpha activity with various physiological and behavioral activity.     

Construction of haptic stimulus patterns for use in cross-modal and intramodal matching tasks

A technique for the construction of haptic stimulus patterns to be used in conjunction with auditory and visual patterns in cross-modal and intramodal matching tasks is described. Special attention is paid to the control of confounding influences arising from the spatiotemporal and the active-passive contrasts.     

Constructing the exact significance level for a person fit statistic

This paper uses an extension of the network algorithm originally introduced by Mehta and Patel to construct exact tail probabilities for testing the general hypothesis that item responses are distributed according to the Rasch model. By assuming that item difficulties are known, the algorithm is applicable to the statistical tests either given the maximum likelihood ability estimate or conditioned on the total score. A simulation study indicates that the network algorithm is an efficient tool for computing the significance level of a person fit statistic based on test lengths of 30 items or less.     

Two distinct patterns of interference in between-attribute Stroop matching tasks

In between-attribute Stroop matching tasks, participants compare the meaning (or the color) of a Stroop stimulus with a probe color (or meaning) while attempting to ignore the Stroop stimulus’s task-irrelevant attribute. Interference in this task has been explained by two competing theories: A semantic competition account and a response competition account. Recent results favor the response competition account, which assumes that interference is caused by a task-irrelevant comparison. However, the comparison of studies is complicated by the lack of a consensus on how trial types should be classified and analyzed. In this work, we review existing findings and theories and provide a new classification of trial types. We report two experiments that demonstrate the superiority of the response competition account in explaining the basic pattern of performance while also revealing its limitations. Two qualitatively distinct interference patterns are identified, resulting from different types of task-irrelevant comparisons. By finding the same interference pattern across task versions, we were additionally able to demonstrate the comparability of processes across two task versions frequently used in neurophysiological and cognitive studies. An integrated account of both types of interference is presented and discussed.     

The underestimation of egocentric distance: evidence from frontal matching tasks

There is controversy over the existence, nature, and cause of error in egocentric distance judgments. One proposal is that the systematic biases often found in explicit judgments of egocentric distance along the ground may be related to recently observed biases in the perceived declination of gaze (Durgin & Li, Attention, Perception, & Psychophysics, in press), To measure perceived egocentric distance nonverbally, observers in a field were asked to position themselves so that their distance from one of two experimenters was equal to the frontal distance between the experimenters. Observers placed themselves too far away, consistent with egocentric distance underestimation. A similar experiment was conducted with vertical frontal extents. Both experiments were replicated in panoramic virtual reality. Perceived egocentric distance was quantitatively consistent with angular bias in perceived gaze declination (1.5 gain). Finally, an exocentric distance-matching task was contrasted with a variant of the egocentric matching task. The egocentric matching data approximate a constant compression of perceived egocentric distance with a power function exponent of nearly 1; exocentric matches had an exponent of about 0.67. The divergent pattern between egocentric and exocentric matches suggests that they depend on different visual cues.