Slope bias of psychometric functions derived from adaptive data

Several investigators have fit psychometric functions to data from adaptive procedures for threshold estimation. Although the threshold estimates are in general quite correct, one encounters a slope bias that has not been explained up to now. The present paper demonstrates slope bias for parametric and nonparametric maximum-likelihood fits and for Spearman-Kärber analysis of adaptive data. The examples include staircase and stochastic approximation procedures. The paper then presents an explanation of slope bias based on serial data dependency in adaptive procedures. Data dependency is first illustrated with simple two-trial examples and then extended to realistic adaptive procedures. Finally, the paper presents an adaptive staircase procedure designed to measure threshold and slope directly. In contrast to classical adaptive threshold-only procedures, this procedure varies both a threshold and a spread parameter in response to double trials.     

Slope bias of psychometric functions derived from adaptive data.

Several investigators have fit psychometric functions to data from adaptive procedures for threshold estimation. Although the threshold estimates are in general quite correct, one encounters a slope bias that has not been explained up to now. The present paper demonstrates slope bias for parametric and nonparametric maximum-likelihood fits and for Spearman-K?rber analysis of adaptive data. The examples include staircase and stochastic approximation procedures. The paper then presents an explanation of slope bias based on serial data dependency in adaptive procedures. Data dependency is first illustrated with simple two-trial examples and then extended to realistic adaptive procedures. Finally, the paper presents an adaptive staircase procedure designed to measure threshold and slope directly. In contrast to classical adaptive threshold-only procedures, this procedure varies both a threshold and a spread parameter in response to double trials.     

Modeling psychometric functions in R

We demonstrate some procedures in the statistical computing environment R for obtaining maximum likelihood estimates of the parameters of a psychometric function by fitting a generalized nonlinear regression model to the data. A feature for fitting a linear model to the threshold (or other) parameters of several psychometric functions simultaneously provides a powerful tool for testing hypotheses about the data and, potentially, for reducing the number of parameters necessary to describe them. Finally, we illustrate procedures for treating one parameter as a random effect that would permit a simplified approach to modeling stimulus-independent variability due to factors such as lapses or interobserver differences. These tools will facilitate a more comprehensive and explicit approach to the modeling of psychometric data.     

Fitting model-based psychometric functions to simultaneity and temporal-order judgment data: MATLAB and R routines

Research on temporal-order perception uses temporal-order judgment (TOJ) tasks or synchrony judgment (SJ) tasks in their binary SJ2 or ternary SJ3 variants. In all cases, two stimuli are presented with some temporal delay, and observers judge the order of presentation. Arbitrary psychometric functions are typically fitted to obtain performance measures such as sensitivity or the point of subjective simultaneity, but the parameters of these functions are uninterpretable. We describe routines in MATLAB and R that fit model-based functions whose parameters are interpretable in terms of the processes underlying temporal-order and simultaneity judgments and responses. These functions arise from an independent-channels model assuming arrival latencies with exponential distributions and a trichotomous decision space. Different routines fit data separately for SJ2, SJ3, and TOJ tasks, jointly for any two tasks, or also jointly for the three tasks (for common cases in which two or even the three tasks were used with the same stimuli and participants). Additional routines provide bootstrap p-values and confidence intervals for estimated parameters. A further routine is included that obtains performance measures from the fitted functions. An R package for Windows and source code of the MATLAB and R routines are available as Supplementary Files.     

Adaptive psychophysical methods for nonmonotonic psychometric functions

Many psychophysical tasks in current use render nonmonotonic psychometric functions; these include the oddball task, the temporal generalization task, the binary synchrony judgment task, and other forms of the same–different task. Other tasks allow for ternary responses and render three psychometric functions, one of which is also nonmonotonic, like the ternary synchrony judgment task or the unforced choice task. In all of these cases, data are usually collected with the inefficient method of constant stimuli (MOCS), because extant adaptive methods are only applicable when the psychometric function is monotonic. This article develops stimulus placement criteria for adaptive methods designed for use with nonmonotonic psychometric functions or with ternary tasks. The methods are transformations of conventional up–down rules. Simulations under three alternative psychophysical tasks prove the validity of these methods, their superiority to MOCS, and the accuracy with which they recover direct estimates of the parameters determining the psychometric functions, as well as estimates of derived quantities such as the point of subjective equality or the difference limen. Practical recommendations and worked-out examples are provided to illustrate how to use these adaptive methods in empirical research.     

Nonparametric tests for equality of psychometric functions

Many empirical studies measure psychometric functions (curves describing how observers’ performance varies with stimulus magnitude) because these functions capture the effects of experimental conditions. To assess these effects, parametric curves are often fitted to the data and comparisons are carried out by testing for equality of mean parameter estimates across conditions. This approach is parametric and, thus, vulnerable to violations of the implied assumptions. Furthermore, testing for equality of means of parameters may be misleading: Psychometric functions may vary meaningfully across conditions on an observer-by-observer basis with no effect on the mean values of the estimated parameters. Alternative approaches to assess equality of psychometric functions per se are thus needed. This paper compares three nonparametric tests that are applicable in all situations of interest: The existing generalized Mantel–Haenszel test, a generalization of the Berry–Mielke test that was developed here, and a split variant of the generalized Mantel–Haenszel test also developed here. Their statistical properties (accuracy and power) are studied via simulation and the results show that all tests are indistinguishable as to accuracy but they differ non-uniformly as to power. Empirical use of the tests is illustrated via analyses of published data sets and practical recommendations are given. The computer code in matlab and R to conduct these tests is available as Electronic Supplemental Material.     

Temporal order and tactile patterns

Temporal order judgments (TOJs) were obtained for tactile stimuli presented to subjects' fingerpads. In one set of measurements, pairs of spatial patterns were presented successively to a single fingerpad (same-site condition), to two fingers on the same hand (ipsilateral condition), or to two fingers on opposite hands (bilateral condition). The subjects were instructed to report which one of the two patterns was presented first. TOJs were more accurate in the same-site condition than in either the ipsilateral or the bilateral conditions. In the ipsilateral and bilateral conditions, performance improved when judging which one of two locations received a stimulus first, although performance levels were still lower than in the same-site condition. Increasing the size of the pattern set from which the two patterns to be judged were drawn had only a slight effect on same-site performance and no effect on ipsilateral/bilateral performance; however, changing the nature of the patterns had a considerable effect on same-site performance and a smaller effect on ipsilateral/bilateral performance. Introducing an intensity imbalance between members of the pair of stimuli also had a large effect on same-site TOJs: a less intense stimulus tended to be judged as being presented first. In the bilateral condition, however, there was a small effect in the reverse direction: more intense stimuli tended to be judged as being presented first. The intensity imbalance had no effect in the ipsilateral condition. The results suggest that different mechanisms are responsible for TOJs for patterns presented to the same-site and to separate sites and, furthermore, that separate sites may constitute separate channels for spatial information.     

Temporal order judgments with amnesia

The hypothesis that patients with anterograde amnesia fail to recognize the temporal relations among recognizable past events was evaluated. Experiment 1 showed that order recognition for words in a list was at chance levels even though event recognition was significantly above chance. The same pattern did not obtain for intact adults. Experiment 2 examined event and order recognition for news events. Amnesics' event recognition for news events was quite accurate and comparable to controls, yet their order recognition was significantly below that of controls, again at chance levels. It is argued that the failure to encode the temporal relations among events has broad consequences for memory performance in general.     

Temporal order judgment in dyslexia

Hari et al. (Brain 174:1373-1380, 2001) demonstrated that dyslexics showed a sluggish attention capture in both visual hemifields. Additionally, they indicated a left-right asymmetry in the perception of temporal order of two visual stimuli (they performed worse than controls only if the stimulus in the left hemifield preceded that in right hemifield). They suggested that a left-sided minineglect is associated with dyslexia. We hypothesized that if a kind of neglect syndrome is responsible for the asymmetry they found, dyslexics should not only show a left-right asymmetry in temporal order judgment of two laterally presented stimuli but also perform equally well as controls when the stimuli are vertically aligned. Our results indicated that in both tasks dyslexics performed generally worse than normal readers. The results suggest that dyslexics suffer from a more general problem of order discrimination.     

Confidence intervals for the parameters of psychometric functions

A Monte Carlo method for computing the bias and standard deviation of estimates of the parameters of a psychometric function such as the Weibull/Quick is described. The method, based on Efron's parametric bootstrap, can also be used to estimate confidence intervals for these parameters. The method's ability to predict bias, standard deviation, and confidence intervals is evaluated in two ways. First, its predictions are compared to the outcomes of Monte Carlo simulations of psychophysical experiments. Second, its predicted confidence intervals were compared with the actual variability of human observers in a psychophysical task. Computer programs implementing the method are available from the author.     

Estimation of psychometric functions from adaptive tracking procedures

Because adaptive tracking procedures are designed to avoid stimulus levels far from a target threshold value, the psychometric function constructed from the trial-by-trial data in the track may be accurate near the target level but a poor reflection of performance at levels far removed from the target. A series of computer simulations was undertaken to assess the reliability and accuracy of psychometric functions generated from data collected in up-down adaptive tracking procedures. Estimates of psychometric function slopes were obtained from-trial-by-trial data in simulated adaptive tracks and compared with the true characteristics of the functions used to generate the tracks. Simulations were carried out for three psychophysical procedures and two target performance levels, with tracks generated by psychometric functions with three different slopes. The functions reconstructed from the tracking data were, for the most part, accurate reflections of the true generating functions when at least 200 trials were included in the tracks. However, for 50- and 100-trial tracks, slope estimates were biased high for all simulated experimental conditions. Correction factors for slope estimates from these tracks are presented. There was no difference in the accuracy and reliability of slope estimation due to -target-level-for the adaptive track, and only minor differences due to psychophysical procedure. It is recommended that, if both threshold and slope of psychometric functions are to be estimated-from the trial-by-trial tracking data, at least 100 trials should be included in the tracks, and a three- or four-alternative forced-choice procedure should be used. However, good estimates can also be obtained using the two-alternative forced-choice procedure or less than 100 trials if appropriate corrections for bias are applied.     

On the form of psychometric functions for taste

Psychometric functions were determined for the discrimination of weak solutions of quinine hydrochloride or of hydrochloric acid from distilled water. The slopes of these functions were compared with those of some functions previously determined for sodium chloride and for sucrose. In general, the functions for QHCl were the least steep and those for NaCl were the most steep, but the differences were not great. Thus, it appears that the psychometric functions for these different taste qualities have substantially the same form, even though absolute sensitivity varies over several orders of magnitude.     

Normative and psychometric data on the body image assessment--revised

After falling into disfavor in the early 1990s, the construct of body image, as measured by body-size estimation (BSE) techniques, has been the focus of increasing interest in the eating disorder literature because of recent theoretical, empirical, and methodological advances. However, no published BSE measure to date has been shown to be psychometrically sound, well normed, inexpensive, and straightforward. This article provides normative and psychometric data for an adapted silhouette BSE measure. Comprehensive normative data are presented on college women's cognitively and affectively based body-size estimates, as well as their desired body size and related discrepancy indexes (cognitive vs. desired, affective vs. desired, affective vs. cognitive). Preliminary data indicate that indexes from the new measure are moderately reliable over time, consistent with their theoretical link to fluctuations in body-related attitudes. Data also support the convergent validity of the measures. Affectively based BSE, alone or as part of a discrepancy measure with desired body size, was most strongly related to measures of eating pathology, body focus, body dissatisfaction, and depressed affect.     

Spatial-frequency uncertainty and cuing effects on psychometric functions for contrast detection

In the present study, the effects of spatial-frequency uncertainty and cuing on psychometric functions for contrast detection of sinusoidal gratings are examined. For this purpose, psychometric functions were collected from 4 subjects under fixed-frequency, randomized-frequency, and cued-frequency conditions. The experiment was conducted with a temporal two-alternative forced-choice task, and five spatial frequencies in the range of 0.5 and 8.0 c/deg and seven contrast levels for each frequency were used. The results showed that the psychometric functions for the randomized-frequency condition were shallower than those for the fixed-frequency condition, supporting the single-band model for the uncertainty effects (Hübner, 1993a, 1993b). For the cued-frequency condition, the slopes of the functions were not clearly different from those for the randomized condition. These results clearly differ from those of Hübner (1996b), which showed, in the spatial two-alternative forced-choice task, steeper psychometric functions for the randomized-frequency condition than those for the fixed- and cued-frequency conditions, supporting the multiple-band model (Hübner, 1993a, 1993b). The difference suggests that the single-band model applies to the uncertainty effects in the temporal forced-choice task, whereas the multiple-band model does so in the spatial forced-choice task.     

Temporal order relations in language comprehension

The role of temporal orientation (chronological or reverse) and chronological distance (close, intermediate, or distant) in general event knowledge on language comprehension was examined. Experiment 1 used a relation-recognition paradigm in which the comprehension of a target event could be facilitated or disrupted by the temporal orientation implied by the prior information. Experiments 2 and 3 used a sentence-probe-recognition paradigm in which the temporal orientation, the stimulus onset asynchrony, and the chronological distance between the sentence event and the probe event were manipulated. The results demonstrated that readers used temporal information conveyed by their knowledge to construct situation models while comprehending sentences. The internal temporal dimension appeared to be directional and reflected the chronological distance between everyday events.     

Solving implicit equations in psychometric data analysis

Many data analysis problems in psychology may be posed conveniently in terms which place the parameters to be estimated on one side of an equation and an expression in these parameters on the other side. A rule for improving the rate of convergence of the iterative solution of such equations is developed and applied to four problems: the principal axis communality problem, individual differences multidimensional scaling,L _P norm multiple regression, andL _P norm factor analysis of a data matrix. The rule results in substantially faster solutions or in solutions where none would be possible without the rule.This work was supported by National Research Council of Canada grant APA 320 to the author.     

Estimation of psychometric functions from adaptive tracking procedures.

Because adaptive tracking procedures are designed to avoid stimulus levels far from a target threshold value, the psychometric function constructed from the trial-by-trial data in the track may be accurate near the target level but a poor reflection of performance at levels far removed from the target. A series of computer simulations was undertaken to assess the reliability and accuracy of psychometric functions generated from data collected in up-down adaptive tracking procedures. Estimates of psychometric function slopes were obtained from trial-by-trial data in simulated adaptive tracks and compared with the true characteristics of the functions used to generate the tracks. Simulations were carried out for three psychophysical procedures and two target performance levels, with tracks generated by psychometric functions with three different slopes. The functions reconstructed from the tracking data were, for the most part, accurate reflections of the true generating functions when at least 200 trials were included in the tracks. However, for 50- and 100-trial tracks, slope estimates were biased high for all simulated experimental conditions. Correction factors for slope estimates from these tracks are presented. There was no difference in the accuracy and reliability of slope estimation due to target level for the adaptive track, and only minor differences due to psychophysical procedure. It is recommended that, if both threshold and slope of psychometric functions are to be estimated from the trial-by-trial tracking data, at least 100 trials should be included in the tracks, and a three- or four-alternative forced-choice procedure should be used. However, good estimates can also be obtained using the two-alternative forced-choice procedure or less than 100 trials if appropriate corrections for bias are applied.     

On the theoretical error bound for estimating psychometric functions

The theoretical limits to the amount of error, or the Cramer-Rao bounds, were derived for estimating psychometric functions. These theoretical error bounds were compared with the variability of psychometric functions estimated from human as well as computer-simulated observers. For the simulated observers, due to the limited efficiency of the sampling strategies, including the placement of the signals and the distribution of the trials, the variances of the estimated parameters are seven times the theoretical bound for threshold and 22 times that for slope. For the human observers, the variance is 18 times the theoretical bounds for threshold and 80 times that for slope. Therefore, a major portion of the variances (60% for threshold and 73% for slope) for the human observers is associated with factors other than sampling strategies. Further improvement of the accuracy for estimating psychometric functions will depend on not only optimizing the sampling strategy, but also better understanding the various sources of error related to the behavior of human observers.