Visual attention and the temporal dynamics of feature integration

Two experiments studied the emergence of bindings between stimulus features (object files) and between stimulus and response features (event files) over time. Choice responses (R2) were signalled by the shape of a stimulus (S2) that followed another stimulus (S1) of the same or different shape, location, and colour. S1 did not require a response (Experiment 1) or trigger a precued simple response (R1) that was or was not repeated by R2 (Experiment 2). Results demonstrate that the mere cooccurrence of stimulus features, and of stimuli and responses, is sufficient to bind their codes. Bindings emerge quickly and remain intact for at least four seconds. Which features are considered depends on their task‐relevance; hence, integration reflects the current attentional set. There was no consistent trend toward higher order interactions as a function of time or of the amount of attention devoted to S1, suggesting that features are not integrated into a single, global superstructure, but enter independent local bindings presumably subserving different functions.     

Actions travel with their objects: evidence for dynamic event files

Moving a visual object is known to lead to an update of its cognitive representation. Given that object representations have also been shown to include codes describing the actions they were accompanied by, we investigated whether these action codes “move” along with their object. We replicated earlier findings that repeating stimulus and action features enhances performance if other features are repeated, but attenuates performance if they alternate. However, moving the objects in which the stimuli appeared in between two stimulus presentations had a strong impact on the feature bindings that involved location. Taken together, our findings provide evidence that changing the location of an object leaves two memory traces, one referring to its original location (an episodic record) and another referring to the new location (a working-memory trace).     

How much attention does an event file need?

Attentional requirements for the spontaneous integration of stimulus and response features were analyzed. In line with previous findings, carrying out a prepared response to the onset of a stimulus created bindings between the response and the features of that stimulus, thereby impairing subsequent performance on mismatching stimulus-response combinations. Findings demonstrate that a stimulus gets bound to a response even if its presence is neither necessary nor useful for the task at hand, it follows rather than precedes the response in time, it competes with a task-relevant stimulus, and if the response is suppressed--but only if the stimulus appears close to the response's eventual execution or abandonment. A multiple-integration model is suggested that assumes that the integration of stimulus features in perception and of response features in action planning are local processes that are independent of stimulus-response integration, which presumably is triggered by the success of the perception-action episode.     

Auditory event files: Integrating auditory perception and action planning

The features of perceived objects are processed in distinct neural pathways, which call for mechanisms that integrate the distributed information into coherent representations (the binding problem). Recent studies of sequential effects have demonstrated feature binding not only in perception, but also across (visual) perception and action planning. We investigated whether comparable effects can be obtained in and across auditory perception and action. The results from two experiments revealed effects indicative of spontaneous integration of auditory features (pitch and loudness, pitch and location), as well as evidence for audio—manual stimulus—response integration. Even though integration takes place spontaneously, features related to task-relevant stimulus or response dimensions are more likely to be integrated. Moreover, integration seems to follow a temporal overlap principle, with features coded close in time being more likely to be bound together. Taken altogether, the findings are consistent with the idea of episodic event files integrating perception and action plans.     

Social priming of hemispatial neglect affects spatial coding: Evidence from the Simon task

In the Simon effect (SE), choice reactions are fast if the location of the stimulus and the response correspond when stimulus location is task-irrelevant; therefore, the SE reflects the automatic processing of space. Priming of social concepts was found to affect automatic processing in the Stroop effect. We investigated whether spatial coding measured by the SE can be affected by the observer’s mental state. We used two social priming manipulations of impairments: one involving spatial processing - hemispatial neglect (HN) and another involving color perception - achromatopsia (ACHM). In two experiments the SE was reduced in the “neglected” visual field (VF) under the HN, but not under the ACHM manipulation. Our results show that spatial coding is sensitive to spatial representations that are not derived from task-relevant parameters, but from the observer’s cognitive state. These findings dispute stimulus-response interference models grounded on the idea of the automaticity of spatial processing.     

The influence of dimensional overlap on location-related priming in the Simon task

Choice reaction times are shorter when stimulus and response locations are compatible than when they are incompatible as in the Simon effect. Recent studies revealed that Simon effects are strongly attenuated when there is temporal overlap with a different high-priority task, accompanied by a decrease of early location-related response priming as reflected in the lateralized readiness potential (LRP). The latter result was obtained in a study excluding overlap of stimulus location with any other dimension in the tasks. Independent evidence suggests that location-related priming might be present in conditions with dimensional overlap. Here we tested this prediction in a dual-task experiment supplemented with recording LRPs. The secondary task was either a standard Simon task where irrelevant stimulus location overlapped with dimensions of the primary task or a Stroop-like Simon task including additional overlap of irrelevant and relevant stimulus attributes. At high temporal overlap, there was no Simon effect nor was there stimulus-related response priming in either condition. Therefore stimulus-triggered response priming seems to be abolished in conditions of limited capacity even if the likelihood of an S–R compatibility effect is maximized.     

Transfer of orthogonal stimulus–response mappings to an orthogonal Simon task

When up–down stimulus locations are mapped to left–right keypresses, an overall advantage for the up–right/down–left mapping is often obtained that varies as a function of response eccentricity. This orthogonal stimulus–response compatibility (SRC) effect also occurs when stimulus location is irrelevant, a phenomenon called the orthogonal Simon effect, and has been attributed to correspondence of stimulus and response code polarities. The Simon effect for horizontal stimulus–response (S–R) arrangements has been shown to be affected by short-term S–R associations established through the mapping used for a prior SRC task in which stimulus location was relevant. We examined whether such associations also transfer between orthogonal SRC and Simon tasks and whether correspondence of code polarities continues to contribute to performance in the Simon task. In Experiment 1, the orthogonal Simon effect was larger after practising with an up–right/down–left mapping of visual stimuli to responses than with the alternative mapping, for which the orthogonal Simon effect tended to reverse. Experiment 2 showed similar results when practice was with high (up) and low (down) pitch tones, though the influence of practice mapping was not as large as that in Experiment 1, implying that the short-term S–R associations acquired in practice are at least in part not modality specific. In Experiment 3, response eccentricity and practice mapping were shown to have separate influences on the orthogonal Simon effect, as expected if both code polarity and acquired S–R associations contribute to performance.     

Inhibition of return for target discriminations: the effect of repeating discriminated and irrelevant stimulus dimensions

Inhibition of return (IOR) refers to slowed reaction times when a target repeats in the same location as a preceding stimulus. In four experiments, the participants were presented with two successive stimuli, S1 and S2. In Experiments 1 and 2, the participants made a speeded discrimination of the identity or orientation of both S1 and S2 (Experiment 1) or of S2 only (Experiment 2). An IOR effect occurred for the repetition of stimulus location, but a facilitatory effect occurred if the stimulus remained unchanged or if an overt response was repeated. In Experiments 3 and 4, the participants localized S1 and S2 (Experiment 3) or S2 only (Experiment 4) to the left or right of center. In this case, repeating the same stimulus had no effect: IOR occurred any time stimulus location repeated. These results demonstrate that the expression of IOR is modulated by the repetition of a target object, but only when the task requires the discrimination of that object; when no discrimination is required, IOR is unaffected.     

S-R compatibility and response selection

In serial stage models, perception and action are usually thought to be linked to each other by an S-R translation mechanism. However, phenomena of S-R compatibility suggest a more direct relationship between perceptual and action domains. We discuss behavioral and psychophysiological evidence that irrelevant stimulus information automatically activates response codes, but then decays over time.

In a series of reaction time studies and electrophysiological experiments, we investigated both temporal and functional properties of the assumed automatic response activation process. We found that the amount of interference due to irrelevant spatial information depends upon how long its availability precedes that of the information relevant for response selection. This indicates that response activation decays rather quickly. If response-relevant and irrelevant spatial information are simultaneously available, electrophysiological measurements show that automatic activation of the spatially corresponding response rises soon after stimulus onset, but then dissipates and gets replaced by the activation of the response indicated by the relevant stimulus attribute.

We conclude that these findings do not support a pure translation account, but rather suggest the presence of two parallel and (at least partially) independent routes from perception to action: A direct route, allowing for automatic activation of response codes if stimulus and response features overlap, and an indirect route linking S and R codes in an arbitrary manner. Via the direct route responses may be primed independent of task-specific contingencies, while the correct response is selected via the indirect route. This use suggests that (a) the transmission of stimulus information to response stages does not (fully) depend on task relevance and that (b) different stimulus features can be transmitted asynchronously and independently from one another.     

You better stop! Binding “stop” tags to irrelevant stimulus features

We investigated whether the basic process of integrating stimuli (and their features) with simultaneously executed responses transfers to situations in which one does not respond to a stimulus. In three experiments, a stop-signal task was combined with a sequential priming paradigm to test whether irrelevant stimulus features become associated with a “stop” tag. Stopping a simple response during the prime trial delayed responding and facilitated stopping in the probe if the same irrelevant stimulus feature was repeated in the probe. These repetition priming effects were independent of the relation between the to-be-executed (or to-be-stopped) responses in the prime and probe, indicating that “stop” tags are global (“stop all responses!”) rather than being response-related (e.g., “stop left response!”).     

Correspondence effects with manual gestures and postures: a study of imitation

In this study, the authors applied methods and theories from research of stimulus-response compatibility (SRC) to action imitation. In 6 experiments, they adopted the logic of the Simon paradigm (B. Hommel & W. Prinz, 1996) to explore interference between task-relevant symbolic stimulus features (color) and task-irrelevant iconic stimulus features (2 hand gestures and 2 postures). The same 2 hand gestures served as responses. Pronounced correspondence effects for both gestures and postures showed up throughout. In line with theories of SRC, the authors account for these correspondence effects in terms of overlap arising between stimulus and response features in a common representational domain. As a specific extension of this approach, they propose 2 functionally independent mechanisms: One operates movement-based when dynamic information is provided, and the other operates state-based with static postures as stimuli. Implications for theories of both SRC and action imitation are discussed.     

S-R compatibility effects due to context-dependent spatial stimulus coding

Responses are faster with spatial S-R correspondence than with noncorrespondence (spatial compatibility effect), even if stimulus location is irrelevant (Simon effect). In two experiments, we sought to determine whether stimuli located above and below a fixation point are coded as left and right (and thus affect the selection of left and right responses) if the visual context suggests such a coding. So, stimuli appeared on the left or right eye of a face’s image that was tilted by 90° to one side or the other (Experiment 1) or varied between upright and 45° or 90° tilting (Experiment 2). Whether stimulus location was relevant (Experiment 1) or not (Experiment 2), responses were faster with correspondence of (face-based) stimulus location and (egocentrically defined) response location, even if stimulus and response locations varied on physically orthogonal dimensions. This suggests that object-based spatial stimulus codes are formed automatically and thus influence the speed of response selection.     

Intermodal event files: integrating features across vision,audition, taction,and action

Understanding how the human brain integrates features of perceived events calls for the examination of binding processes within and across different modalities and domains. Recent studies of feature-repetition effects have demonstrated interactions between shape, color, and location in the visual modality and between pitch, loudness, and location in the auditory modality: repeating one feature is beneficial if other features are also repeated, but detrimental if not. These partial-repetition costs suggest that co-occurring features are spontaneously bound into temporary event files. Here, we investigated whether these observations can be extended to features from different sensory modalities, combining visual and auditory features in Experiment 1 and auditory and tactile features in Experiment 2. The same types of interactions, as for unimodal feature combinations, were obtained including interactions between stimulus and response features. However, the size of the interactions varied with the particular combination of features, suggesting that the salience of features and the temporal overlap between feature-code activations plays a mediating role.     

When the voluntary mind meets the irresistible event: Stimulus–response correspondence effects on task selection during voluntary task switching

In the present study, we investigated how task selection is biased by inherent stimulus characteristics in the voluntary task-switching paradigm. We used digits as the task stimuli, since they may automatically induce spatially horizontal representations of numbers. Specifically, we examined whether an irrelevant spatial representation of a number coincides with its associated response codes and whether such a stimulus–response (S–R) correspondence effect biases task selection for a digit. Participants were randomly assigned to one of two different action code layout conditions: Two numerical tasks were arranged as inner/outer in the horizontal layout condition or as upper/down in the vertical layout condition. Participants in the horizontal layout condition were more likely to choose a task when the task’s action code and the digit’s spatial representation corresponded, as compared with when they did not. On the other hand, no selection bias was observed in the vertical layout condition, since there was no overlapping spatial representation between the stimulus and response. The present study extends previous findings by considering the influence of the stimulus-driven effect on task selection with regard to the S–R correspondence effect.     

Interference From Multi-dimensional Objects During Feature and Conjunction Discriminations

Feature discrimination performance within an attended object and interference from irrelevant, multi-dimensional objects (distractors) were examined in a two-choice, response compatibility paradigm. Results showed that the amount of interference by multi-dimensional distractors was dependent on three factors: (1) the discriminability of the incompatible, task-relevant distractor features; (2) the number of incompatible, task-relevant distractor features; and (3) whether the task-relevant, incompatible features matched the task goals. The most interesting finding was that additive priming effects were found for multiple, task-relevant features that matched the task goals, whether these features were present in the attended object or in the ignored object. Models that assume that each task-relevant feature primes its corresponding decision/response asynchronously and that this priming is combined to meet a decision/response criterion (at least when attended) can account for distractor interference during conjunction discriminations. Implications of these findings for feature integration models, template models, and a response selection model are discussed.     

Inhibitory tagging in inhibition of return: Evidence from flanker interference with multiple distractor features

Fuentes, Vivas, and Humphreys (1999) proposed a distinction betweeninhibition of return (IOR) and inhibitory processing taking place at a location subject to IOR. This latter mechanism,inhibitory tagging (IT), would act at a late level of processing related to response selection. In the present study, we examined whether IT was applied only to the target-relevant properties of the stimuli (e.g., to its color) or whether it was applied to other features of the stimulus as well (e.g., to its shape). Both when the task was to respond to the target’s color (Experiment 1) as well as when it was to respond to the target’s shape (Experiment 2), there was evidence of IT (reversal of the typical flanker effect at the cued location, relative to the uncued location) only to task-relevant features of the target. These findings suggest that IT is a central process of control constrained by task demands and current goals.     

Trial after trial: General processing consequences as a function of repetition and change in multidimensional sound

While there are pointers relating to the consequences of repetition, a general framework regarding the cognitive implications of processing multidimensional stimuli as a function of previous stimulus history is currently lacking. Three experiments using sounds varying in location and pitch were carried out, in which the immediate consequences of repeating or changing task-relevant and task-irrelevant attributes were orthogonally examined. A consistent pattern of data was shown, in that the magnitude of selective attention failure was larger when the task-relevant value repeated across trials, while differences between dimensions were larger when the task-relevant value changed across trials. These effects of irrelevance and dimension as a function of intertrial contingency are summarized in a model depicting the dynamic allocation of processing resource.     

Processing of irrelevant location information under dual-task conditions

This study deals with the problem of whether the processing of irrelevant location information in Simon-like tasks is triggered exogenously or endogenously. In Experiment 1, the primary task required one to press, as fast as possible, a left-hand-side key or a right-hand-side key (R1) to the pitch of a tone that was presented binaurally (S1). The secondary task required identifying, without time constraints, a visual stimulus (S2) that appeared randomly to the left or right of screen center. Results showed that there was a correspondence (i.e., a cross-task Simon effect) between the location of R1 and the location of S2 when S2 was presented alone. The cross-task Simon effect became much smaller (and in-significant) when a noise stimulus was presented contralateral to S2. Experiment 2 was similar to Experiment 1, except that S2 appeared unpredictably in only one-third of the trials. Results of Experiment 2 closely replicated those of Experiment 1: the cross-task Simon effect was much greater when S2 was presented alone. Experiment 3 differed from Experiment 1 because S2 had to be processed in only one-third of the trials, in which its identity was to be reported. In the remaining two-thirds of the trials, participants could ignore S2. Results confirmed that the cross-task Simon effect was much greater when S2 was presented alone. In contrast, it did not matter whether S2 had to be processed or not. In conclusion, the present study supports the hypothesis that the task-irrelevant spatial code of the stimulus is formed automatically, likely through an exogenously triggered selection. The role of endogenously initiated selection, if any, is much less important.     

Transfer of response codes from choice-response to go/no-go tasks

The present study investigated the conditions for observing the Simon effect in go/no-go tasks. The Simon effect denotes faster and more accurate responses when irrelevant stimulus location and response location correspond than when they do not correspond. In four experiments, participants performed both in a choice-response task (CRT) and in a go/no-go task, and we varied the order and the similarity of the tasks. In the CRT, participants pressed a left key to one stimulus colour and a right key to another stimulus colour; in the go/no-go task, participants pressed one (e.g., left) key to one stimulus colour and refrained from responding to the other stimulus colour. As expected, Simon effects were consistently observed in the CRT. In contrast, Simon effects in the go/no-go task were only observed when it followed the CRT and when the mapping of stimulus colours to response locations was preserved between tasks (i.e., in Experiment 4). Results suggest that transfer of a particular S–R rule including response location from the CRT to the go/no-go task was responsible for the Simon effect in the latter task. In general, results are consistent with a response-discrimination account of the Simon effect.     

An action sequence withheld in memory can delay execution of visually guided actions: the generalization of response compatibility interference

Withholding an action plan in memory for later execution can delay execution of another action, if the actions share a similar (compatible) action feature (i.e., response hand). This phenomenon, termed compatibility interference (CI), was found for identity-based actions that do not require visual guidance. The authors examined whether CI can generalize to both identity-based and location-based actions that require visual guidance. Participants withheld a planned action based on the identity of a stimulus and then immediately executed a visually guided action (touch response) to a 2nd stimulus based on its color identity (Experiment 1), its spatial location (Experiment 2), or an intrinsic spatial location within an object (Experiment 3). Results showed CI for both left- and right-hand responses in Experiment 1. However, CI occurred for left- but not right-hand responses in Experiment 2 and 3. This suggests that CI can generalize to visually guided actions under cognitive control but not to actions that invoke automatic visual-control mechanisms where the left hemisphere may play a special role (C. Gonzalez, T. Ganel, & M. Goodale, 2006). The code occupation account for CI (G. Stoet & B. Hommel, 2002) is also discussed.