Fitting model-based psychometric functions to simultaneity and temporal-order judgment data: MATLAB and R routines

Research on temporal-order perception uses temporal-order judgment (TOJ) tasks or synchrony judgment (SJ) tasks in their binary SJ2 or ternary SJ3 variants. In all cases, two stimuli are presented with some temporal delay, and observers judge the order of presentation. Arbitrary psychometric functions are typically fitted to obtain performance measures such as sensitivity or the point of subjective simultaneity, but the parameters of these functions are uninterpretable. We describe routines in MATLAB and R that fit model-based functions whose parameters are interpretable in terms of the processes underlying temporal-order and simultaneity judgments and responses. These functions arise from an independent-channels model assuming arrival latencies with exponential distributions and a trichotomous decision space. Different routines fit data separately for SJ2, SJ3, and TOJ tasks, jointly for any two tasks, or also jointly for the three tasks (for common cases in which two or even the three tasks were used with the same stimuli and participants). Additional routines provide bootstrap p-values and confidence intervals for estimated parameters. A further routine is included that obtains performance measures from the fitted functions. An R package for Windows and source code of the MATLAB and R routines are available as Supplementary Files.     

Temporal order judgment criteria are affected by synchrony judgment sensitivity

Synchrony perception for audio-visual stimulus pairs is typically studied by using temporal order judgment (TOJ) or synchrony judgment (SJ) tasks. Research has shown that estimates of the point of subjective simultaneity (PSS) obtained using these two methods do not necessarily correspond. Here, we investigate the hypothesis that the PSS estimate obtained in a TOJ task is shifted in the direction of the most sensitive part of the synchrony judgment curve, as obtained in an SJ task. The results confirm that criterion shifts in the TOJ task move the PSS toward regions of the audio-visual temporal interval continuum where discriminations are most sensitive.     

Somatosensory prior entry assessed with temporal order judgments and simultaneity judgments

Attention is central to perception, yet a clear understanding of how attention influences the latency of perception has proven surprisingly elusive. Recent research has indicated that spatially attended stimuli are perceived earlier than unattended stimuli across a range of sensory modalities-an effect termed prior entry. However, the method commonly used to measure this, the temporal order judgment (TOJ) task, has been criticized as susceptible to response bias, despite deliberate attempts to minimize such bias. A preferred alternative is the simultaneity judgment (SJ) task. We tested the prior-entry hypothesis for somatosensory stimuli using both a TOJ task (replicating an earlier experiment) and an SJ task. Prior-entry effects were found for both, though the effect was reduced in the SJ task. Additional experiments (TOJ and SJ) using visual cues established that the earlier perception of cued tactile targets does not result from intramodal sensory interactions between tactile cues and targets.     

Temporal, but not spatial, context modulates a masked prime's effect on temporal order judgment, but not on response latency

Three experiments investigated a dissociation originally described by Neumann, Esselmann, and Klotz. Stimuli were geometric shapes, preceded by similar shapes that were masked by metacontrast. Each experiment consisted of three parts. In the reaction time (RT) part, participants saw an array of geometric shapes, one of which was marked by bars, and had to respond to the marked shape's position by pressing an appropriate button. A prime (a similar, but smaller stimulus) preceded either the marked or an unmarked stimulus. In the temporal order judgment (TOJ) part, the task was to judge the temporal order of the marked and the unmarked stimulus. In the detection part, detectability of the prime was tested. Although its detectability was zero or close to zero, the prime affected both RT and the apparent onset as measured by TOJ. The effect on RT was significantly larger than the effect on TOJ (Exp. 1). Increasing the spatial context (number of non-target stimuli in the display) did not affect this pattern (Exp. 2). By contrast, reducing the temporal context (range of stimulus onset asynchronies) abolished the prime's effect in the TOJ task, although the prime affected RT under identical conditions. It is concluded that partially different mechanisms mediate the prime's effect in the two tasks and that the effect of stimulus context on TOJ found in the Neumann et al. study was due to temporal, not spatial context. Received: 17 December 1997 / Accepted: 15 December 1998     

Audiovisual synchrony and temporal order judgments: effects of experimental method and stimulus type

When an audio-visual event is perceived in the natural environment, a physical delay will always occur between the arrival of the leading visual component and that of the trailing auditory component. This natural timing relationship suggests that the point of subjective simultaneity (PSS) should occur at an auditory delay greater than or equal to 0 msec. A review of the literature suggests that PSS estimates derived from a temporal order judgment (TOJ) task differ from those derived from a synchrony judgment (SJ) task, with (unnatural) auditory-leading PSS values reported mainly for the TOJ task. We report data from two stimulus types that differed in terms of complexity--namely, (1) a flash and a click and (2) a bouncing ball and an impact sound. The same participants judged the temporal order and synchrony of both stimulus types, using three experimental methods: (1) a TOJ task with two response categories ("audio first" or "video first"), (2) an SJ task with two response categories ("synchronous" or "asynchronous"; SJ2), and (3) an SJ task with three response categories ("audio first," "synchronous," or "video first"; SJ3). Both stimulus types produced correlated PSS estimates with the SJ tasks, but the estimates from the TOJ procedure were uncorrelated with those obtained from the SJ tasks. These results suggest that the SJ task should be preferred over the TOJ task when the primary interest is i n perceived audio-visualsynchrony.     

Dissociations between reaction times and temporal order judgments: a diffusion model approach

A diffusion model for simple reaction time (RT) and temporal order judgment (TOJ) tasks was developed to account for a commonly observed dissociation between these 2 tasks: Most stimulus manipulations (e.g., intensity) have larger effects in RT tasks than in TOJ tasks. The model assumes that a detection criterion determines the level of sensory evidence needed to conclude that a stimulus has been presented. Analysis of the performance that would be achieved with different possible criterion settings revealed that performance was optimal with a lower criterion setting for the TOJ task than for the RT task. In addition, the model predicts that effects of stimulus manipulations should increase with the size of the detection criterion. Thus, the model suggests that commonly observed dissociations between RT and TOJ tasks may simply be due to performance optimization in the face of conflicting task demands.     

Differential effects of visual-spatial attention on response latency and temporal-order judgment

Summary Theorists from both classical structuralism and modern attention research have claimed that attention to a sensory stimulus enhances processing speed. However, they have used different operations to measure this effect, viz., temporal-order judgment (TOJ) and reaction-time (RT) measurement. We report two experiments that compared the effect of a spatial cue on RT and TOJ. Experiment 1 demonstrated that a nonmasked, peripheral cue (the brief brightening of a box) affected both RT and TOJ. However, the former effect was significantly larger than the latter. A masked cue had a smaller, but reliable, effect on TOJ. In Experiment 2, the effects of a masked cue on RT and TOJ were compared under identical stimulus conditions. While the cue had a strong effect on RT, it left TOJ unaffected. These results suggest that a spatial cue may have dissociable effects on response processes and the processes that lead to a conscious percept. Implications for the concept of direct parameter specification and for theories of visual attention are discussed.     

Perception of Temporal Order: The Effects of Age,Sex, and Cognitive Factors

ABSTRACT

The present paper investigates the effects of age, sex, and cognitive factors on temporal-order perception. Nine temporal-order tasks were employed using two and four stimuli presented in the auditory and visual modalities. Significantly increased temporal-order thresholds (TOT) in the elderly were found for almost all tasks, while sex differences were only observed for two tasks. Multiple regression analyses show that the performance on most temporal-order tasks can be predicted by cognitive factors, such as speed of fluid reasoning, short-term memory, and attention. However, age was a significant predictor of TOT in three tasks using visual stimuli. We conclude (1) that age-related differences can often be attributed to cognitive factors involved in temporal-order perception, and (2) that the concept of temporal-order perception is more complex than implied by the current models.     

Prior-entry: A review

The law of prior entry was one of E.B. Titchener’s seven fundamental laws of attention. According to Titchener (1908, p. 251): “the object of attention comes to consciousness more quickly than the objects which we are not attending to.” Although researchers have been studying prior entry for more than a century now, progress in understanding the effect has been hindered by the many methodological confounds present in early research. As a consequence, it is unclear whether the behavioral effects reported in the majority of published studies in this area should be attributed to attention, decisional response biases, and/or, in the case of exogenous spatial cuing studies of the prior-entry effect, to sensory facilitation effects instead. In this article, the literature on the prior-entry effect is reviewed, the confounds present in previous research highlighted, current consensus summarized, and some of the key questions for future research outlined. In particular, recent research has now provided compelling psychophysical and electrophysiological evidence to support the claim that attending to a sensory modality, spatial location, or stimulus feature/attribute can all give rise to a relative speeding-up of the time of arrival of attended, as compared to relatively less attended (or unattended) stimuli. Prior-entry effects have now been demonstrated following both the endogenous and exogenous orienting of attention, though prior-entry effects tend to be smaller in magnitude when assessed by means of participants’ performance on SJ tasks than when assessed by means of their performance on TOJ tasks.     

触觉时序知觉手臂交叉效应的性别差异

本研究探讨触觉时序知觉的手臂交叉效应是否存在性别差异。通过两个实验在较短和较长的SOA条件下考察男性和女性被试在基于体觉和基于外部空间的触觉时序判断任务中的表现。结果表明,男性与女性被试在基于体觉和基于外部空间的触觉时序判断任务中均存在手臂交叉效应,SOA较短时男性被试的手臂交叉效应显著小于女性被试,但在SOA较长的条件下手臂交叉效应没有明显的性别差异。触觉时序知觉手臂交叉效应的性别差异可能与空间知觉能力和生理解剖学因素有关。     

Task-irrelevant sounds influence both temporal order and apparent-motion judgments about tactile stimuli applied to crossed and uncrossed hands

It has been suggested that judgments about the temporal–spatial order of successive tactile stimuli depend on the perceived direction of apparent motion between them. Here we manipulated tactile apparent-motion percepts by presenting a brief, task-irrelevant auditory stimulus temporally in-between pairs of tactile stimuli. The tactile stimuli were applied one to each hand, with varying stimulus onset asynchronies (SOAs). Participants reported the location of the first stimulus (temporal order judgments: TOJs) while adopting both crossed and uncrossed hand postures, so we could scrutinize skin-based, anatomical, and external reference frames. With crossed hands, the sound improved TOJ performance at short (≤300 ms) and at long (>300 ms) SOAs. When the hands were uncrossed, the sound induced a decrease in TOJ performance, but only at short SOAs. A second experiment confirmed that the auditory stimulus indeed modulated tactile apparent motion perception under these conditions. Perceived apparent motion directions were more ambiguous with crossed than with uncrossed hands, probably indicating competing spatial codes in the crossed posture. However, irrespective of posture, the additional sound tended to impair potentially anatomically coded motion direction discrimination at a short SOA of 80 ms, but it significantly enhanced externally coded apparent motion perception at a long SOA of 500 ms. Anatomically coded motion signals imply incorrect TOJ responses with crossed hands, but correct responses when the hands are uncrossed; externally coded motion signals always point toward the correct TOJ response. Thus, taken together, these results suggest that apparent-motion signals are likely taken into account when tactile temporal–spatial information is reconstructed.

     

判断框架对时序知觉位置启动效应的影响

研究采用包含两种判断框架（判断刺激出现的先后次序）的时序判断任务探讨了决策偏向对时序知觉位置启动效应的影响。实验结果表明,时序知觉中存在着位置启动效应,而且在两个靶刺激出现时间间隔不同时,被试在“哪个靶刺激先出现”和“哪个靶刺激后出现”两种判断框架下的正确率存在显著差异,但在两个靶刺激同时出现时并没有出现显著的判断框架效应。因此,本研究推测时序知觉位置启动效应的作用机制不仅包括知觉增强加工过程,而且也与认知决策有关,这为反应偏向理论提供了新的支持证据。     

The role of tone duration in dichotic temporal order judgment

Over the past two decades, a number of studies have been based on the dichotic temporal order judgment (TOJ) paradigm, to compare auditory temporal processing in various subpopulations to that of young, normal controls. The reported estimates of dichotic TOJ thresholds, expressed as the interstimulus intervals (ISIs) for 75 % accuracy among the controls, have varied. In the present study, we examined the influence of tone duration, within the 10- to 40-msec range, on dichotic TOJ accuracy and threshold. The results indicated that increases in either ISI or tone duration increased dichotic TOJ accuracy similarly, implying that changes in tone duration may affect dichotic TOJs simply by adding to the delay between onset of the tone at the lead ear and onset of the tone at the lag ear. The dichotic TOJ thresholds from three published studies that had used tones as stimuli and the dichotic thresholds from the present study all fell within 0.5 standard deviations from the theoretical line of a 10-msec reduction in threshold for every 10-msec increase in tone duration. When the dichotic TOJ threshold data from the present study and from the published studies were converted to stimulus onset asynchronies (SOAs) and plotted as a function of tone duration, they fell very close to or on a zero-slope line, indicating that when ISI and duration are summed to yield an SOA, dichotic TOJ thresholds are invariant to tone duration within the range of 10–40 msec.     

A model-based comparison of three theories of audiovisual temporal recalibration

Observers change their audio-visual timing judgements after exposure to asynchronous audiovisual signals. The mechanism underlying this temporal recalibration is currently debated. Three broad explanations have been suggested. According to the first, the time it takes for sensory signals to propagate through the brain has changed. The second explanation suggests that decisional criteria used to interpret signal timing have changed, but not time perception itself. A final possibility is that a population of neurones collectively encode relative times, and that exposure to a repeated timing relationship alters the balance of responses in this population. Here, we simplified each of these explanations to its core features in order to produce three corresponding six-parameter models, which generate contrasting patterns of predictions about how simultaneity judgements should vary across four adaptation conditions: No adaptation, synchronous adaptation, and auditory leading/lagging adaptation. We tested model predictions by fitting data from all four conditions simultaneously, in order to assess which model/explanation best described the complete pattern of results. The latency-shift and criterion-change models were better able to explain results for our sample as a whole. The population-code model did, however, account for improved performance following adaptation to a synchronous adapter, and best described the results of a subset of observers who reported least instances of synchrony.     

Investigating the effects of inversion on configural processing with an audiovisual temporal-order judgment task

Research has shown that inversion is more detrimental to the perception of faces than to the perception of other types of visual stimuli. Inverting a face results in an impairment of configural information processing that leads to slowed early face processing and reduced accuracy when performance is tested in face recognition tasks. We investigated the effects of inverting speech and non-speech stimuli on audiovisual temporal perception. Upright and inverted audiovisual video clips of a person uttering syllables (experiments 1 and 2), playing musical notes on a piano (experiment 3), or a rhesus monkey producing vocalisations (experiment 4) were presented. Participants made unspeeded temporal-order judgments regarding which modality stream (auditory or visual) appeared to have been presented first. Inverting the visual stream did not have any effect on the sensitivity of temporal discrimination responses in any of the four experiments, thus implying that audiovisual temporal integration is resilient to the effects of orientation in the picture plane. By contrast, the point of subjective simultaneity differed significantly as a function of orientation only for the audiovisual speech stimuli but not for the non-speech stimuli or monkey calls. That is, smaller auditory leads were required for the inverted than for the upright-visual speech stimuli. These results are consistent with the longer processing latencies reported previously when human faces are inverted and demonstrates that the temporal perception of dynamic audiovisual speech can be modulated by changes in the physical properties of the visual speech (ie by changes in orientation).     

Latency facilitation in temporal-order judgments: time course of facilitation as a function of judgment type

The paper is concerned with two models of early visual processing which predict that priming of a visual mask by a preceding masked stimulus speeds up conscious perception of the mask (perceptual latency priming). One model ascribes this speed-up to facilitation by visuo-spatial attention [Scharlau, I., & Neumann, O. (2003a). Perceptual latency priming by masked and unmasked stimuli: Evidence for an attentional explanation. Psychological Research 67, 184-197], the other attributes it to nonspecific upgrading mediated by retino-thalamic and thalamo-cortical pathways [Bachmann, T. (1994). Psychophysiology of visual masking: The fine structure of conscious experience. Commack, NY: Nova Science Publishers]. The models make different predictions about the time course of perceptual latency priming. Four experiments test these predictions. The results provide more support for the attentional than for the upgrading model. The experiments further demonstrate that testing latency facilitation with temporal-order judgments may induce a methodological problem resulting in fairly low estimates. A method which provides a more exhaustive measure is suggested and tested.     

The bisection point across variants of the task

Bisection tasks are used in research on normal space and time perception and to assess the perceptual distortions accompanying neurological disorders. Several variants of the bisection task are used, which often yield inconsistent results, prompting the question of which variant is most dependable and which results are to be trusted. We addressed this question using theoretical and experimental approaches. Theoretical performance in bisection tasks is derived from a general model of psychophysical performance that includes sensory components and decisional processes. The model predicts how performance should differ across variants of the task, even when the sensory component is fixed. To test these predictions, data were collected in a within-subjects study with several variants of a spatial bisection task, including a two-response variant in which observers indicated whether a line was transected to the right or left of the midpoint, a three-response variant (which included the additional option to respond “midpoint”), and a paired-comparison variant of the three-response format. The data supported the model predictions, revealing that estimated bisection points were least dependable with the two-response variant, because this format confounds perceptual and decisional influences. Only the three-response paired-comparison format can separate out these influences. Implications for research in basic and clinical fields are discussed.     

Audiotactile interactions in temporal perception

In the present review, we focus on how commonalities in the ontogenetic development of the auditory and tactile sensory systems may inform the interplay between these signals in the temporal domain. In particular, we describe the results of behavioral studies that have investigated temporal resolution (in temporal order, synchrony/asynchrony, and simultaneity judgment tasks), as well as temporal numerosity perception, and similarities in the perception of frequency across touch and hearing. The evidence reviewed here highlights features of audiotactile temporal perception that are distinctive from those seen for other pairings of sensory modalities. For instance, audiotactile interactions are characterized in certain tasks (e.g., temporal numerosity judgments) by a more balanced reciprocal influence than are other modality pairings. Moreover, relative spatial position plays a different role in the temporal order and temporal recalibration processes for audiotactile stimulus pairings than for other modality pairings. The effect exerted by both the spatial arrangement of stimuli and attention on temporal order judgments is described. Moreover, a number of audiotactile interactions occurring during sensory-motor synchronization are highlighted. We also look at the audiotactile perception of rhythm and how it may be affected by musical training. The differences emerging from this body of research highlight the need for more extensive investigation into audiotactile temporal interactions. We conclude with a brief overview of some of the key issues deserving of further research in this area.     

Perception of intersensory synchrony in audiovisual speech: not that special

Perception of intersensory temporal order is particularly difficult for (continuous) audiovisual speech, as perceivers may find it difficult to notice substantial timing differences between speech sounds and lip movements. Here we tested whether this occurs because audiovisual speech is strongly paired (“unity assumption”). Participants made temporal order judgments (TOJ) and simultaneity judgments (SJ) about sine-wave speech (SWS) replicas of pseudowords and the corresponding video of the face. Listeners in speech and non-speech mode were equally sensitive judging audiovisual temporal order. Yet, using the McGurk effect, we could demonstrate that the sound was more likely integrated with lipread speech if heard as speech than non-speech. Judging temporal order in audiovisual speech is thus unaffected by whether the auditory and visual streams are paired. Conceivably, previously found differences between speech and non-speech stimuli are not due to the putative “special” nature of speech, but rather reflect low-level stimulus differences.     

Increased visual bias in children with developmental coordination disorder: Evidence from a visual-tactile temporal order judgment task

BackgroundPrevious studies have suggested that children with developmental coordination disorder (DCD) rely heavily on vision to perform movements, which may contribute to their clumsy movements. However, few studies have objectively and quantitatively investigated the perceptual biases of children with DCD.MethodsA visual-tactile temporal order judgment (TOJ) task was used to measure and compare the perceptual biases of 19 children with DCD and 19 age- and sex-matched typically developing children. The point of subjective equality, which demonstrates when “visual first” and “tactile first” judgment probabilities are equal (50%), obtained by analyzing the results of the visual-tactile TOJ task, was used as an indicator of perceptual biases. Further, variables (age and manual dexterity in all participants; motor function, autism spectrum disorder and attention-deficit hyperactivity disorder traits, and depressive symptoms in children with DCD) associated with perceptual biases were examined with correlation analysis.ResultsChildren with DCD had significantly stronger visual bias than typically developing children. Overall correlation analysis showed that increased visual bias was significantly correlated with poor manual dexterity.ConclusionChildren with DCD had a strong visual bias, which was associated with poor manual dexterity.