Attentional control of the creation and retrieval of stimulus–response bindings

Two experiments studied the degree to which the creation and retrieval of episodic feature bindings is modulated by attentional control. Experiment 1 showed that the impact of bindings between stimulus and response features varies as a function of the current attentional set: only bindings involving stimulus features that match the current set affect behavior. Experiment 2 varied the time point at which new attentional sets were implemented—either before or after the processing of the to-be-integrated stimuli and responses. The time point did not matter, suggesting that the attentional set has no impact on feature integration proper but controls which features get access to and can thus trigger the retrieval of bindings.     

Feature integration across perception and action: event files affect response choice

Five experiments investigated the spontaneous integration of stimulus and response features. Participants performed simple, prepared responses (R1) to the mere presence of Go signals (S1) before carrying out another, freely chosen response (R2) to another stimulus (S2), the main question being whether the likelihood of repeating a response depends on whether or not the stimulus, or some of its features, are repeated. Indeed, participants were more likely to repeat the previous response if stimulus form or color was repeated than if it was alternated. The same was true for stimulus location, but only if location was made task-relevant, whether by defining the response set in terms of location, by requiring the report of S2 location, or by having S1 to be selected against a distractor. These findings suggest that task-relevant stimulus and response features are spontaneously integrated into independent, local event files, each linking one stimulus to one response feature. Upon reactivation of one member of the binary link activation is spread to the other, thereby increasing the likelihood to repeat a response if one or more stimulus features are repeated. These findings support the idea that both perceptual events and action plans are cognitively represented in terms of their features, and that feature-integration processes cross borders between perception and action.     

Temporal dynamics of unimodal and multimodal feature binding

In two experiments, we studied the temporal dynamics of feature integration with auditory (Experiment 1) and audiovisual (Experiment 2) stimuli and manual responses. Consistent with previous observations, performance was better when the second of two consecutive stimuli shared all or none of the features of the first, rather than when only one of the features overlapped. Comparable partial-overlap costs were obtained for combinations of stimulus features and responses. These effects decreased systematically with increasing time between the two stimulus-and-response events, and the decreased rate was comparable for unimodal and multimodal bindings. General effect size reflected the degree of task relevance of the dimension or modality of the respective feature, but the effects of relevance and of temporal delay did not interact. This suggests that the processing of stimuli on task-relevant sensory modalities and feature dimensions is facilitated by task-specific attentional sets, whereas the temporal dynamics might reflect that bindings “decay” or become more difficult to access over time.     

Priming and binding in and across perception and action: A correlational analysis of the internal structure of event files

Individual performance was compared across three different tasks that tap into the binding of stimulus features in perception, the binding of action features in action planning, and the emergence of stimulus–response bindings (“event files”). Within a task correlations between the size of binding effects were found within visual perception (e.g., the strength of shape–location binding correlated positively with the strength of shape–colour binding) but not between perception and action planning, suggesting different, domain-specific binding mechanisms. To some degree, binding strength was predicted by priming effects of the respective features, especially if these features varied on a dimension that matched the current attentional set.     

Transfer of orthogonal stimulus–response mappings to an orthogonal Simon task

When up–down stimulus locations are mapped to left–right keypresses, an overall advantage for the up–right/down–left mapping is often obtained that varies as a function of response eccentricity. This orthogonal stimulus–response compatibility (SRC) effect also occurs when stimulus location is irrelevant, a phenomenon called the orthogonal Simon effect, and has been attributed to correspondence of stimulus and response code polarities. The Simon effect for horizontal stimulus–response (S–R) arrangements has been shown to be affected by short-term S–R associations established through the mapping used for a prior SRC task in which stimulus location was relevant. We examined whether such associations also transfer between orthogonal SRC and Simon tasks and whether correspondence of code polarities continues to contribute to performance in the Simon task. In Experiment 1, the orthogonal Simon effect was larger after practising with an up–right/down–left mapping of visual stimuli to responses than with the alternative mapping, for which the orthogonal Simon effect tended to reverse. Experiment 2 showed similar results when practice was with high (up) and low (down) pitch tones, though the influence of practice mapping was not as large as that in Experiment 1, implying that the short-term S–R associations acquired in practice are at least in part not modality specific. In Experiment 3, response eccentricity and practice mapping were shown to have separate influences on the orthogonal Simon effect, as expected if both code polarity and acquired S–R associations contribute to performance.     

Retrieval of bindings between task-irrelevant stimuli and responses can facilitate behaviour under conditions of high response certainty

Repetition priming can be driven by the encoding and retrieval of stimulus–response (S–R) bindings. When a previously encoded S–R binding is retrieved, and is congruent with the response currently required, it can bias response-selection processes towards selecting the retrieved response, resulting in facilitation. Previous studies have used classification tasks at retrieval. Here, two (or more) response options are competing, and it is likely that any evidence (e.g., an S–R binding) in favour of one option will be utilized to effect a decision. Thus, S–R effects are likely to be seen when using such a task. It is unclear whether such effects can be seen under conditions of higher response certainty, when participants are explicitly cued to make a response. Across two experiments, evidence for a modulating influence of S–R bindings is seen despite using a response cueing method at retrieval to minimize response uncertainty and despite stimuli being task irrelevant. Finally, the results suggest that responses within these S–R bindings are coded at the level of left versus right hand, and not a more fine-grained within-hand thumb versus index finger. The results underline the resilience of S–R effects, suggesting that they are present even under conditions where no explicit object-oriented decision is required.     

Stimulus–response translation in manual and vocal modality: Is age‐related slowing influenced by changes in working memory functioning?

In some conditions, age‐related slowing rates are higher for manual than for vocal responses. The interaction between age and response modality was examined in four experiments in which the performance of younger (M = 22.6 years) and older (M = 70.8 years) adults was compared under various stimulus–response mapping rules. The interaction effect was not observed in a simple localisation task on which the manual modality had an advantage in both age groups (Experiment 1). When congruence between stimulus and response was reduced (Experiment 2), manual responses were disproportionately slowed in older participants. In conceptual and size comparison tasks (Experiment 3), the age‐by‐modality effect was not influenced by working memory load during the interval between the first and the second stimulus presentation. In an identification task, age‐related slowing effects were less when responses were spoken names than when nonwords were used as labels (Experiment 4). These results indicate that older participants experience a specific difficulty in activating arbitrary S–R mapping rules during executive control.     

Processing of irrelevant location information under dual-task conditions

This study deals with the problem of whether the processing of irrelevant location information in Simon-like tasks is triggered exogenously or endogenously. In Experiment 1, the primary task required one to press, as fast as possible, a left-hand-side key or a right-hand-side key (R1) to the pitch of a tone that was presented binaurally (S1). The secondary task required identifying, without time constraints, a visual stimulus (S2) that appeared randomly to the left or right of screen center. Results showed that there was a correspondence (i.e., a cross-task Simon effect) between the location of R1 and the location of S2 when S2 was presented alone. The cross-task Simon effect became much smaller (and in-significant) when a noise stimulus was presented contralateral to S2. Experiment 2 was similar to Experiment 1, except that S2 appeared unpredictably in only one-third of the trials. Results of Experiment 2 closely replicated those of Experiment 1: the cross-task Simon effect was much greater when S2 was presented alone. Experiment 3 differed from Experiment 1 because S2 had to be processed in only one-third of the trials, in which its identity was to be reported. In the remaining two-thirds of the trials, participants could ignore S2. Results confirmed that the cross-task Simon effect was much greater when S2 was presented alone. In contrast, it did not matter whether S2 had to be processed or not. In conclusion, the present study supports the hypothesis that the task-irrelevant spatial code of the stimulus is formed automatically, likely through an exogenously triggered selection. The role of endogenously initiated selection, if any, is much less important.     

The influence of visual noise in the binding of irrelevant features to responses

ABSTRACT

In order to efficiently control our actions, stimuli and responses made to them are stored and integrated into event files. This integration is not restricted to relevant stimuli. Even irrelevant stimuli or features that co-occur at responding can be integrated with the response. In the presence of only one irrelevant feature, binding effects for that feature are generally observed. In the presence of more than one irrelevant features, empirical evidence is inconsistent. The present experiments tested feature-response binding effects for irrelevant features when more than one irrelevant feature was present. In two experiments feature-response binding effects for three irrelevant features (colour, shape, and location) were tested in a block-wise manner. In each block one feature was orthogonally varied to the response while the others were held constant (Experiment 1) or always changed (Experiment 2). Significant binding effects were observed for each of three features when the other two were held constant (Exp. 1). However, when the other two features were varied, significant binding effects were only observed for colour and location, but not for shape (Exp. 2). The results suggest that strength of feature-response binding effects is influenced by the variation of simultaneously presented features.     

Feature binding in visual short-term memory is unaffected by task-irrelevant changes of location,shape, and color

Three experiments used a change detection paradigm across a range of study–test intervals to address the respective contributions of location, shape, and color to the formation of bindings of features in sensory memory and visual short-term memory (VSTM). In Experiment 1, location was designated task irrelevant and was randomized between study and test displays. The task was to detect changes in the bindings between shape and color. In Experiments 2 and 3, shape and color, respectively, were task irrelevant and randomized, with bindings tested between location and color (Experiment 2) and location and shape (Experiment 3). At shorter study–test intervals, randomizing location was most disruptive, followed by shape and then color. At longer intervals, randomizing any task-irrelevant feature had no impact on change detection for bindings between features, and location had no special role. Results suggest that location is crucial for initial perceptual binding but loses that special status once representations are formed in VSTM, which operates according to different principles, than do visual attention and perception.     

Visual form recognition threshold and the psychological refractory period

Two experiments studied the effect of a reaction time response (RT) on visual form recognition threshold when the temporal interval separating the RT stimulus and the recognition stimulus was short. In Experiment 1 an initial RT response to an auditory signal did not impair the subsequent forced-choice visual form recognition threshold. Interstimulus intervals (ISI) of 0, 50, 100, 150, and 200 msec were used; S was always aware of the ISI under test. In Experiment 2 a visual stimulus was used to elicit the R T response; this shift to an intramodal stimulus did not alter the recognition threshold. These data were interpreted as supporting the hypothesis that two stimulus events can be processed simultaneously even when the temporal interval between them is short.     

Logical recoding of S-R rules can reverse the effects of spatial S-R correspondence

Two experiments investigated competing explanations for the reversal of spatial stimulus—response (S—R) correspondence effects (i.e., Simon effects) with an incompatible S—R mapping on the relevant, nonspatial dimension. Competing explanations were based on generalized S—R rules (logical-recoding account) or referred to display—control arrangement correspondence or to S—S congruity. In Experiment 1, compatible responses to finger—name stimuli presented at left/right locations produced normal Simon effects, whereas incompatible responses to finger—name stimuli produced an inverted Simon effect. This finding supports the logical-recoding account. In Experiment 2, spatial S—R correspondence and color S—R correspondence were varied independently, and main effects of these variables were observed. The lack of an interaction between these variables, however, disconfirms a prediction of the display—control arrangement correspondence account. Together, the results provide converging evidence for the logical-recoding account. This account claims that participants derive generalized response selection rules (e.g., the identity or reversal rule) from specific S—R rules and inadvertently apply the generalized rules to the irrelevant (spatial) S—R dimension when selecting their response.     

Flanker and Simon effects interact at the response selection stage

The present study aimed at investigating the processing stage underlying stimulus–stimulus (S–S) congruency effects by examining the relation of a particular type of congruency effect (i.e., the flanker effect) with a stimulus–response (S–R) spatial correspondence effect (i.e., the Simon effect). Experiment 1 used a unilateral flanker task in which the flanker also acted as a Simon-like accessory stimulus. Results showed a significant S–S Congruency × S–R Correspondence interaction: An advantage for flanker–response spatially corresponding trials was observed in target–flanker congruent conditions, whereas, in incongruent conditions, there was a noncorresponding trials' advantage. The analysis of the temporal trend of the correspondence effects ruled out a temporal-overlap account for the observed interaction. Moreover, results of Experiment 2, in which the flanker did not belong to the target set, demonstrated that this interaction cannot be attributed to perceptual grouping of the target–flanker pairs and referential coding of the target with respect to the flanker in the congruent and incongruent conditions, respectively. Taken together, these findings are consistent with a response selection account of congruency effects: Both the position and the task-related attribute of the flanker would activate the associated responses. In noncorresponding-congruent trials and corresponding-incongruent trials, this would cause a conflict at the response selection stage.     

Stimulus–response bindings contribute to item switch costs in working memory

In counter updating tasks, responses are typically faster when items repeat than when they change (item switch costs). The present study explored the contribution of stimulus–response bindings to these item switch costs. In two experiments, we orthogonally manipulated the repetition/switch of to-be-counted items and the repetition/switch of required manual responses. Item switch costs were considerably lower when item switches were accompanied by response switches than when accompanied by response repetitions. Experiment 2 showed that, although there was also a smaller contribution from stimulus–stimulus bindings (i.e., shape-location), the major part was due to stimulus-response bindings. These results show that in the widely used standard version of the counter updating task, a considerable portion of item switch costs is caused by the unbinding of stimulus–response bindings rather than by processes of switching items in working memory.     

Stimulus–response bindings code both abstract and specific representations of stimuli: evidence from a classification priming design that reverses multiple levels of response representation

Repetition priming can be caused by the rapid retrieval of previously encoded stimulus–response (S–R) bindings. S–R bindings have recently been shown to simultaneously code multiple levels of response representation, from specific Motor-actions to more abstract Decisions (“yes”/”no”) and Classifications (e.g., “man-made”/”natural”). Using an experimental design that reverses responses at all of these levels, we assessed whether S–R bindings also code multiple levels of stimulus representation. Across two experiments, we found effects of response reversal on priming when switching between object pictures and object names, consistent with S–R bindings that code stimuli at an abstract level. Nonetheless, the size of this reversal effect was smaller for such across-format (e.g., word–picture) repetition than for within-format (e.g., picture–picture) repetition, suggesting additional coding of format-specific stimulus representations. We conclude that S–R bindings simultaneously represent both stimuli and responses at multiple levels of abstraction.     

Inhibition of return for target discriminations: the effect of repeating discriminated and irrelevant stimulus dimensions

Inhibition of return (IOR) refers to slowed reaction times when a target repeats in the same location as a preceding stimulus. In four experiments, the participants were presented with two successive stimuli, S1 and S2. In Experiments 1 and 2, the participants made a speeded discrimination of the identity or orientation of both S1 and S2 (Experiment 1) or of S2 only (Experiment 2). An IOR effect occurred for the repetition of stimulus location, but a facilitatory effect occurred if the stimulus remained unchanged or if an overt response was repeated. In Experiments 3 and 4, the participants localized S1 and S2 (Experiment 3) or S2 only (Experiment 4) to the left or right of center. In this case, repeating the same stimulus had no effect: IOR occurred any time stimulus location repeated. These results demonstrate that the expression of IOR is modulated by the repetition of a target object, but only when the task requires the discrimination of that object; when no discrimination is required, IOR is unaffected.     

Binding time: Evidence for integration of temporal stimulus features

Several lines of evidence suggest that during processing of events, the features of these events become connected via episodic bindings. Such bindings have been demonstrated for a large number of visual and auditory stimulus features, like color and orientation, or pitch and loudness. Importantly, most visual and auditory events typically also involve temporal features, like onset time or duration. So far, however, whether temporal stimulus features are also bound into event representations has never been tested directly. The aim of the present study was to investigate possible binding between stimulus duration and other features of auditory events. In Experiment 1, participants had to respond with two keys to a low or high pitch sinus tone. Critically, the tones were presented with two different presentation durations. Sequential analysis of RT data indicated binding of stimulus duration into the event representation: at pitch repetitions, performance was better when both pitch and duration repeated, relative to when only pitch repeated and duration switched. This finding was replicated with loudness as relevant stimulus feature in Experiment 2. In sum, the results demonstrate that temporal features are bound into auditory event representations. This finding is an important advancement for binding theory in general, and raises several new questions for future research.     

Co-occurrence of sequential and practice effects in the Simon task: Evidence for two independent mechanisms affecting response selection

The Simon effect refers to the observation that responses to a relevant stimulus dimension are faster and more accurate when the stimulus and response spatially correspond than when they do not, even though stimulus position is irrelevant. Recent findings have suggested that the Simon effect can be strongly modulated by prior practice with a spatially incompatible mapping and by correspondence sequence. Although practice is thought to influence conditional stimulus —response (S-R) processing, leaving response priming through the unconditional route unaffected, sequential effects are thought to represent trial-by-trial adaptations that selectively involve unconditional S —R processing. In the present study, we tested this assumption by assessing the effects of correspondence sequence both when the Simon task alone was performed and when it was preceded by a spatial compatibility task with either incompatible (Experiments 1-2) or compatible (Experiment 2) instructions. The observation that practice and correspondence sequence co-occur and exert additive effects strongly demonstrates that the two factors affect different processing routes.     

S-R compatibility effects due to context-dependent spatial stimulus coding

Responses are faster with spatial S-R correspondence than with noncorrespondence (spatial compatibility effect), even if stimulus location is irrelevant (Simon effect). In two experiments, we sought to determine whether stimuli located above and below a fixation point are coded as left and right (and thus affect the selection of left and right responses) if the visual context suggests such a coding. So, stimuli appeared on the left or right eye of a face’s image that was tilted by 90° to one side or the other (Experiment 1) or varied between upright and 45° or 90° tilting (Experiment 2). Whether stimulus location was relevant (Experiment 1) or not (Experiment 2), responses were faster with correspondence of (face-based) stimulus location and (egocentrically defined) response location, even if stimulus and response locations varied on physically orthogonal dimensions. This suggests that object-based spatial stimulus codes are formed automatically and thus influence the speed of response selection.     

Response force in RT tasks: Isolating effects of stimulus probability and response probability

Mattes, UIrich, and Miller (1997) found that as response probability decreases in a simple reaction time (RT) task, participants produce more forceful responses as well as longer RTs, suggesting a direct influence of preparatory processes on the motor system. In this previous study, however, response probability was confounded with stimulus probability, leaving open the possibility that response force was sensitive to stimulus- rather than response-related preparation. The present study was conducted to unravel the effects of stimulus and response probability. Experiment 1 manipulated stimulus probability and revealed that responses to a more probable stimulus are less forceful than responses to a less probable stimulus even when both stimuli require the same response. Experiment 2 demonstrated that this stimulus probability effect does not depend on the overall level of response probability. Experiment 3 showed an analogous effect for response probability when stimulus probability is kept constant. The complete pattern of results suggests that both stimulus probability and response probability affect the forcefulness of a response. It is argued that response probability exerts adirect influence on the motor system, whereas stimulus probability influences the motor system indirectly via premotoric adjustments.