A Truncated-Probit Item Response Model for Estimating Psychophysical Thresholds

Human abilities in perceptual domains have conventionally been described with reference to a threshold that may be defined as the maximum amount of stimulation which leads to baseline performance. Traditional psychometric links, such as the probit, logit, and t, are incompatible with a threshold as there are no true scores corresponding to baseline performance. We introduce a truncated probit link for modeling thresholds and develop a two-parameter IRT model based on this link. The model is Bayesian and analysis is performed with MCMC sampling. Through simulation, we show that the model provides for accurate measurement of performance with thresholds. The model is applied to a digit-classification experiment in which digits are briefly flashed and then subsequently masked. Using parameter estimates from the model, individuals’ thresholds for flashed-digit discrimination is estimated.     

A computer program for Spearman-Kärber and probit analysis of psychometric function data

PMETRIC is a computer program for the analysis of observed psychometric functions. It can estimate the parameters of these functions, using either probit analysis (a parametric technique) or the Spearman-Kärber method (a nonparametric one). For probit analysis, either a maximum likelihood or a minimum χ² criterion may be used for parameter estimation. In addition, standard errors of parameter estimates can be estimated via bootstrapping. The program can be used to analyze data obtained from either yes-no orm-alternative forced-choice tasks. To facilitate the use of PMETRIC in simulation work, an associated program, PMETGEN, is provided for the generation of simulated psychometric function data. Use of PMETRIC is illustrated with data from a duration discrimination task.     

Discriminative performance of the domestic hen in a visual acuity task

The performance of 6 domestic hens on a visual acuity task was studied using a spatial discrete-trial conditional discrimination procedure. Gray stimuli and vertical square-wave gratings, ranging in spatial frequency from 1 to 10 cycles per millimeter, were presented in a descending and ascending series of probes. On each trial, either a grating or gray stimulus was presented. Only one spatial frequency of grating was presented during each session. Between probe sessions, training continued at the coarsest grating value. Stimulus discriminability, measured as values of log d, changed with changes in grating spatial frequency for both probe series. Fitted lines described the linear portion of the psychometric functions. Thresholds estimated from the lines generally ranged from four to six cycles per degree, with slightly greater estimates from data from the descending probe series. There were no systematic changes in response bias as a function of grating spatial frequency.     

Psychophysics with children: Investigating the effects of attentional lapses on threshold estimates

When assessing the perceptual abilities of children, researchers tend to use psychophysical techniques designed for use with adults. However, children’s poorer attentiveness might bias the threshold estimates obtained by these methods. Here, we obtained speed discrimination threshold estimates in 6- to 7-year-old children in UK Key Stage 1 (KS1), 7- to 9-year-old children in Key Stage 2 (KS2), and adults using three psychophysical procedures: QUEST, a 1-up 2-down Levitt staircase, and Method of Constant Stimuli (MCS). We estimated inattentiveness using responses to “easy” catch trials. As expected, children had higher threshold estimates and made more errors on catch trials than adults. Lower threshold estimates were obtained from psychometric functions fit to the data in the QUEST condition than the MCS and Levitt staircases, and the threshold estimates obtained when fitting a psychometric function to the QUEST data were also lower than when using the QUEST mode. This suggests that threshold estimates cannot be compared directly across methods. Differences between the procedures did not vary significantly with age group. Simulations indicated that inattentiveness biased threshold estimates particularly when threshold estimates were computed as the QUEST mode or the average of staircase reversals. In contrast, thresholds estimated by post-hoc psychometric function fitting were less biased by attentional lapses. Our results suggest that some psychophysical methods are more robust to attentiveness, which has important implications for assessing the perception of children and clinical groups.

     

On the analysis of psychometric functions: The Spearman-Kärber method

With computer simulations, we examined the performance of the Spearman-Kärber method for analyzing psychometric functions and compared this method with the standard technique of probit analysis. The Spearman-Kärber method was found to be superior in most cases. It generally yielded less biased and less variable estimates of the location and dispersion of a psychometric function, and it provided more power to detect differences in these parameters across experimental conditions. Moreover, the Spearman-Kärber method provided information about the skewness and higher moments of psychometric functions that is beyond the scope of probit analysis. These advantages of the Spearman-Kärber method suggest that it should often be used in preference to probit analysis for the analysis of observed psychometric functions.     

Evaluation of maximum-likelihood estimators in nonintensive auditory psychophysics

This is a brief report on the use of maximum-likelihood (ML) estimators in auditory psychophysics. Slope parameters of psychometric functions are characterized for three nonintensive auditory tasks: forced-choice discrimination of interaural time differences (ΔITD), frequency (Δf), and duration (Δt). Using these slope estimates, the ML method is implemented and threshold estimates are obtained for the three tasks and compared with previously published data. ΔITD thresholds were additionally measured for human observers by means of two other psychophysical procedures: the constant-stimuli (CS) and the 2-down 1-up methods (Wetherill & Levitt, 1965). Standard errors were smallest for the ML method. Finally, simulations showed ML estimates to be more efficient than the CS andk-down 1-up procedures fork=2 to 5. For up—down procedures, efficiency was highest fork values of 3 and 4. The entropy (Shannon, 1949) of ML estimates was the smallest of the simulated procedures, but poorer than ideal by 0.5 bits.     

A computer program for Spearman-K?rber and probit analysis of psychometric function data.

PMETRIC is a computer program for the analysis of observed psychometric functions. It can estimate the parameters of these functions, using either probit analysis (a parametric technique) or the Spearman-K?rber method (a nonparametric one). For probit analysis, either a maximum likelihood or a minimum chi 2 criterion may be used for parameter estimation. In addition, standard errors of parameter estimates can be estimated via bootstrapping. The program can be used to analyze data obtained from either yes-no or m-alternative forced-choice tasks. To facilitate the use of PMETRIC in simulation work, an associated program, PMETGEN, is provided for the generation of simulated psychometric function data. Use of PMETRIC is illustrated with data from a duration discrimination task.     

The visual acuity of the pigeon for distant targets

The acuity of six pigeons was measured in an apparatus that required the birds to make visual discriminations at a distance of 28.75 in. (73 cm) from the stimulus targets. The stimuli were black and white gratings of varying stripe width. A forced choice procedure was used, and both the Method of Constant Stimuli and a descending series technique determined the order of stimulus presentation. Thresholds, obtained by interpolating at the 25% error point on the psychometric functions, ranged from 1.16 to 4.0 min of arc. Thresholds measured in the same apparatus for two human observers were 0.79 and 0.82 min of arc. The descending series design produced lower per cent error rates at the widest stripe value, but otherwise there appeared to be no difference between psychophysical methods. Position preferences occurred in most of the birds; differential per cent error functions and differential latency functions to the two keys illustrate these. Retinal histology revealed shallow, centrally located foveae in the three pigeon breeds used.     

Converting between measures of slope of the psychometric function

The psychometric function’s slope provides information about the reliability of psychophysical threshold estimates. Furthermore, knowing the slope allows one to compare, across studies, thresholds that were obtained at different performance criterion levels. Unfortunately, the empirical validation of psychometric function slope estimates is hindered by the bewildering variety of slope measures that are in use. The present article provides conversion formulas for the most popular cases, including the logistic, Weibull, Quick, cumulative normal, and hyperbolic tangent functions as analytic representations, in both linear and log coordinates and to different log bases, the practical decilog unit, the empirically based interquartile range measure of slope, and slope in a? representation of performance.     

Threshold estimation in two-alternative forced-choice (2AFC) tasks: the Spearman-Kärber method

The Spearman-K?rber method can be used to estimate the threshold value or difference limen in two-alternative forced-choice tasks. This method yields a simple estimator for the difference limen and its standard error, so that both can be calculated with a pocket calculator. In contrast to previous estimators, the present approach does not require any assumptions about the shape of the true underlying psychometric function. The performance of this new nonparametric estimator is compared with the standard technique of probit analysis. The Spearman-K?rber method appears to be a valuable addition to the toolbox of psychophysical methods, because it is most accurate for estimating the mean (i.e., absolute and difference thresholds) and dispersion of the psychometric function, although it is not optimal for estimating percentile-based parameters of this function.     

Measuring, estimating, and understanding the psychometric function: A commentary

The psychometric function, relating the subject’s response to the physical stimulus, is fundamental to psychophysics. This paper examines various psychometric function topics, many inspired by this special symposium issue ofPerception & Psychophysics: What are the relative merits of objective yes/no versus forced choice tasks (including threshold variance)? What are the relative merits of adaptive versus constant stimuli methods? What are the relative merits of likelihood versus up-down staircase adaptive methods? Is 2AFC free of substantial bias? Is there no efficient adaptive method for objective yes/no tasks? Should adaptive methods aim for 90% correct? Can adding more responses to forced choice and objective yes/no tasks reduce the threshold variance? What is the best way to deal with lapses? How is the Weibull function intimately related to thed’ function? What causes bias in the likelihood goodness-of-fit? What causes bias in slope estimates from adaptive methods? How good are nonparametric methods for estimating psychometric function parameters? Of what value is the psychometric function slope? How are various psychometric functions related to each other? The resolution of many of these issues is surprising.     

Thresholds for the detection of changing visual features

Change blindness refers to the difficulty observers have in detecting otherwise obvious changes to visual stimuli, when these changes are masked in some way. Typically, change blindness is studied by using complex visual scenes and complex changes to these scenes. In the current study, we used a more controlled visual environment, presenting observers with a series of oriented, sinusoidal patterns (Gabors), one of which underwent a change during a blanking of the screen. Changes were made to different features (size, colour, spatial frequency, and speed) with the target-distractor discriminability varying. The detectability of these changes was quantified by calculating psychometric functions and thresholds for each individual observer. Thresholds for the detection of changing features were higher than those for non-changing features, but thresholds for both tasks show consistency across observers. Psychometric-function slopes show consistency across observers and change type only for non-changing targets. For changing targets, psychometric-function slopes show no obvious pattern across observers or change types. We suggest this reflects vSTM treating different features as abstract, interchangeable tokens, as alternative explanations (such as additional noise in vSTM) can be ruled out.     

The biasing effect of personality on self-estimates of cognitive abilities in males and females

In addition to exploring the relationship between psychometric and self-estimated measures of Crystallized Knowledge (Gc) and Visual Processing (Gv), this study investigated whether personality significantly moderated these relationships, thereby influencing the accuracy of the self-estimates. Adult participants (N = 165) completed the Big Five Inventory and self-estimated their levels of Gc and Gv. They were subsequently administered the Multidimensional Aptitude Battery II, a group test of cognitive ability. Significant and positive relationships between psychometric Gc and Gv and their respective self-estimates were found. Additionally, investigation of the moderating effects of personality for each gender separately using standard multiple regressions found that females high in Extraversion and males low in Conscientiousness were more prone to overestimating their Gv ability, while males high in Openness provided more accurate estimates of their Gv than those low in Openness. Elucidating the personality traits that distort self-perceptions of intellectual functioning has significant implications for the identification of individuals at risk of harboring inaccurate expectations, leading to the potential for interventions aimed at ameliorating associated deleterious consequences.     

The application of probit analysis to the results of mental tests

The application of the Müller-Urban constant process to item selection, as considered in a recent paper in this journal, is shown to be closely analogous to a method now in general use for the analysis of insecticidal and other toxicological tests. This method ofprobit analysis gives the maximum likelihood estimates of the unknown parameters and, with the aid of published tables, the necessary computations can be rapidly performed. The present paper contains an outline of the method and an illustration of the most convenient form of computations for use in analyses of psychometric data.     

Trend in correlated proportions

A random effects probit model is developed for the case in which the same units are sampled repeatedly at each level of an independent variable. Because the observed proportions may be correlated under these conditions, estimating their trend with respect to the independent variable is no longer a standard problem for probit, logit or loglinear analysis. Using a qualitative analogue of a random regressions model, we employ instead marginal maximum likelihood to estimate the average latent trend line. Likelihood ratio tests of the hypothesis of no trend in the average line, and the hypothesis of no differences in average trend lines between experimental treatments, are proposed. We illustrate the model both with simulated data and with observed data from a clinical experiment in which psychiatric patients on two drug therapies are rated on five occasions for the presence or absence of symptoms.Supported by a grant from the MacArthur Foundation and National Science Foundation Grant BNS85-11774.The authors are indebted to James Heckman for calling our attention to the Clark algorithm.     

Adaptive psychophysical methods for nonmonotonic psychometric functions

Many psychophysical tasks in current use render nonmonotonic psychometric functions; these include the oddball task, the temporal generalization task, the binary synchrony judgment task, and other forms of the same–different task. Other tasks allow for ternary responses and render three psychometric functions, one of which is also nonmonotonic, like the ternary synchrony judgment task or the unforced choice task. In all of these cases, data are usually collected with the inefficient method of constant stimuli (MOCS), because extant adaptive methods are only applicable when the psychometric function is monotonic. This article develops stimulus placement criteria for adaptive methods designed for use with nonmonotonic psychometric functions or with ternary tasks. The methods are transformations of conventional up–down rules. Simulations under three alternative psychophysical tasks prove the validity of these methods, their superiority to MOCS, and the accuracy with which they recover direct estimates of the parameters determining the psychometric functions, as well as estimates of derived quantities such as the point of subjective equality or the difference limen. Practical recommendations and worked-out examples are provided to illustrate how to use these adaptive methods in empirical research.     

Bayesian active probabilistic classification for psychometric field estimation

Psychometric functions are typically estimated by fitting a parametric model to categorical subject responses. Procedures to estimate unidimensional psychometric functions (i.e., psychometric curves) have been subjected to the most research, with modern adaptive methods capable of quickly obtaining accurate estimates. These capabilities have been extended to some multidimensional psychometric functions (i.e., psychometric fields) that are easily parameterizable, but flexible procedures for general psychometric field estimation are lacking. This study introduces a nonparametric Bayesian psychometric field estimator operating on subject queries sequentially selected to improve the estimate in some targeted way. This estimator implements probabilistic classification using Gaussian processes trained by active learning. The accuracy and efficiency of two different actively sampled estimators were compared to two non-actively sampled estimators for simulations of one of the simplest psychometric fields in common use: the pure-tone audiogram. The actively sampled methods achieved estimate accuracy equivalent to the non-actively sampled methods with fewer observations. This trend held for a variety of audiogram phenotypes representative of the range of human auditory perception. Gaussian process classification is a general estimation procedure capable of extending to multiple input variables and response classes. Its success with a two-dimensional psychometric field informed by binary subject responses holds great promise for extension to complex perceptual models currently inaccessible to practical estimation.

     

The psychometric function: I. Fitting, sampling, and goodness of fit

The psychometric function relates an observer’s performance to an independent variable, usually some physical quantity of a stimulus in a psychophysical task. This paper, together with its companion paper (Wichmann & Hill, 2001), describes an integrated approach to (1) fitting psychometric functions, (2) assessing the goodness of fit, and (3) providing confidence intervals for the function’s parameters and other estimates derived from them, for the purposes of hypothesis testing. The present paper deals with the first two topics, describing a constrained maximum-likelihood method of parameter estimation and developing several goodness-of-fit tests. Using Monte Carlo simulations, we deal with two specific difficulties that arise when fitting functions to psychophysical data. First, we note that human observers are prone to stimulus-independent errors (orlapses). We show that failure to account for this can lead to serious biases in estimates of the psychometric function’s parameters and illustrate how the problem may be overcome. Second, we note that psychophysical data sets are usually rather small by the standards required by most of the commonly applied statistical tests. We demonstrate the potential errors of applying traditionalX ² methods to psychophysical data and advocate use of Monte Carlo resampling techniques that do not rely on asymptotic theory. We have made available the software to implement our methods.     

Perceptual learning in monocular pattern masking: experiments and explanations by the twin summation gain control model of contrast processing

We investigated practice effects on contrast thresholds for target patterns. Results showed that practice decreased contrast thresholds when targets were presented on maskers. Thresholds tended to decrease more at the higher end of the masker contrast range. At least partially, learning transferred to stimuli of the untrained phase. We simulated changes in threshold versus contrast functions using a contrast-processing model and then fit the model to pre- and posttraining data. The simulation results and model fit suggest that learning in pattern masking can be accounted for by changes in nonlinear transducer functions for divisive inhibitory signals.     

Converting between measures of slope of the psychometric function.

The psychometric function's slope provides information about the reliability of psychophysical threshold estimates. Furthermore, knowing the slope allows one to compare, across studies, thresholds that were obtained at different performance criterion levels. Unfortunately, the empirical validation of psychometric function slope estimates is hindered by the bewildering variety of slope measures that are in use. The present article provides conversion formulas for the most popular cases, including the logistic, Weibull, Quick, cumulative normal, and hyperbolic tangent functions as analytic representations, in both linear and log coordinates and to different log bases, the practical decilog unit, the empirically based interquartile range measure of slope, and slope in a d' representation of performance.