Hearing what the body feels: auditory encoding of rhythmic movement

Phillips-Silver and Trainor (Phillips-Silver, J., Trainor, L.J., (2005). Feeling the beat: movement influences infants' rhythm perception. Science, 308, 1430) demonstrated an early cross-modal interaction between body movement and auditory encoding of musical rhythm in infants. Here we show that the way adults move their bodies to music influences their auditory perception of the rhythm structure. We trained adults, while listening to an ambiguous rhythm with no accented beats, to bounce by bending their knees to interpret the rhythm either as a march or as a waltz. At test, adults identified as similar an auditory version of the rhythm pattern with accented strong beats that matched their previous bouncing experience in comparison with a version whose accents did not match. In subsequent experiments we showed that this effect does not depend on visual information, but that movement of the body is critical. Parallel results from adults and infants suggest that the movement-sound interaction develops early and is fundamental to music processing throughout life.     

Do you see what I see? Affect and visual information processing

Individuals in sad moods process information in a less global and more local manner than do those in happier moods. This experiment investigates whether processing speed is associated with these mood effects, whether task ambiguity moderates these mood effects, and whether making feelings appear irrelevant to the task can eliminate these mood effects. Participants in happy, sad, and neutral moods were lead to experience their feelings as being either relevant or irrelevant to a global/local processing task. As predicted, sad moods decreased global processing relative to happier moods when feelings seemed relevant to the task and when the criteria for responding were ambiguous, but not when feelings seemed irrelevant or when the criteria were unambiguous. Consistent with the idea that mood guides processing, increases in affect intensity were associated with faster reaction times. Overall, the results suggest that mood and processing effects share some core similarities with mood and judgement effects.     

Auditory dominance in temporal processing: new evidence from synchronization with simultaneous visual and auditory sequences

Evidence that audition dominates vision in temporal processing has come from perceptual judgment tasks. This study shows that this auditory dominance extends to the largely subconscious processes involved in sensorimotor coordination. Participants tapped their finger in synchrony with auditory and visual sequences containing an event onset shift (EOS), expected to elicit an involuntary phase correction response (PCR), and also tried to detect the EOS. Sequences were presented in unimodal and bimodal conditions, including one in which auditory and visual EOSs of opposite sign coincided. Unimodal results showed greater variability of taps, smaller PCRs, and poorer EOS detection in vision than in audition. In bimodal conditions, variability of taps was similar to that for unimodal auditory sequences, and PCRs depended more on auditory than on visual information, even though attention was always focused on the visual sequences.     

Identification of temporal order within auditory sequences

Unpracticed Ss reported the order of sounds in sequences consisting of either three or four successive items repeated over and over without pause. With unrelated sounds each lasting 200 msec, correct reports of order were at chance level for oral responses and for card-ordering responses (each card bearing the name of one sound). The sequences with four unrelated items were studied in greater detail, and the threshold for identification was found to be 670 msec with oral responses and 300 msec with card-ordering responses. When two related sounds (tones) were used in four-item sequences, correct card-ordering was possible at 200 msec per item when the tones were temporally contiguous, but was not possible at this duration when the tones were separated by nonrelated sounds. Some special rules governing auditory sequence identification were suggested, and implications for theories of auditory perception discussed.     

See what you want to see: motivational influences on visual perception

People's motivational states--their wishes and preferences--influence their processing of visual stimuli. In 5 studies, participants shown an ambiguous figure (e.g., one that could be seen either as the letter B or the number 13) tended to report seeing the interpretation that assigned them to outcomes they favored. This finding was affirmed by unobtrusive and implicit measures of perception (e.g., eye tracking, lexical decision tasks) and by experimental procedures demonstrating that participants were aware only of the single (usually favored) interpretation they saw at the time they viewed the stimulus. These studies suggest that the impact of motivation on information processing extends down into preconscious processing of stimuli in the visual environment and thus guides what the visual system presents to conscious awareness.     

Modality-specific auditory and visual temporal processing deficits

We studied the attentional blink (AB) and the repetition blindness (RB) effects using an audio-visual presentation procedure designed to overcome several potential methodological confounds in previous cross-modal research. In Experiment 1, two target digits were embedded amongst letter distractors in two concurrent streams (one visual and the other auditory) presented from the same spatial location. Targets appeared in either modality unpredictably at different temporal lags, and the participants' task was to recall the digits at the end of the trial. We evaluated both AB and RB for pairs of targets presented in either the same or different modalities. Under these conditions both AB and RB were observed in vision, AB but not RB was observed in audition, and there was no evidence of AB or RB cross-modally from audition to vision or vice versa. In Experiment 2, we further investigated the AB by including Lag 1 items and observed Lag 1 sparing, thus ruling out the possibility that the observed effects were due to perceptual and/or conceptual masking. Our results support a distinction between a modality-specific interference at the attentional selection stage and a modality-independent interference at later processing stages. They also provide a new dissociation between the AB and RB.     

Action enhances auditory but not visual temporal sensitivity

People naturally dance to music, and research has shown that rhythmic auditory stimuli facilitate production of precisely timed body movements. If motor mechanisms are closely linked to auditory temporal processing, just as auditory temporal processing facilitates movement production, producing action might reciprocally enhance auditory temporal sensitivity. We tested this novel hypothesis with a standard temporal-bisection paradigm, in which the slope of the temporal-bisection function provides a measure of temporal sensitivity. The bisection slope for auditory time perception was steeper when participants initiated each auditory stimulus sequence via a keypress than when they passively heard each sequence, demonstrating that initiating action enhances auditory temporal sensitivity. This enhancement is specific to the auditory modality, because voluntarily initiating each sequence did not enhance visual temporal sensitivity. A control experiment ruled out the possibility that tactile sensation associated with a keypress increased auditory temporal sensitivity. Taken together, these results demonstrate a unique reciprocal relationship between auditory time perception and motor mechanisms. As auditory perception facilitates precisely timed movements, generating action enhances auditory temporal sensitivity.     

Body movement enhances the extraction of temporal structures in auditory sequences

Auditory and motor systems interact in processing auditory rhythms. This study investigated the effect of intuitive body movement, such as head nodding or foot tapping, on listeners’ ability to entrain to the pulse of an auditory sequence. A pulse-finding task was employed using an isochronous sequence of tones in which tones were omitted at pseudorandom positions. Musicians and non-musicians identified their subjectively fitting pulse either using periodic body movement or through listening only. The identified pulse was measured subsequently by finger tapping. Movement appeared to assist pulse extraction especially for non-musicians. The chosen pulse tempi tended to be faster with movement. Additionally, movement led to higher synchronization stabilities of the produced pulse along the sequence, regardless of musical training. These findings demonstrated the facilitatory role of body movement in entraining to auditory rhythms and its interaction with musical training.     

Infants' perception of rhythm: categorization of auditory sequences by temporal structure

We presented 7- to 9-month-old infants with repetitions of three- or four-tone sequences characterized by a particular rhythmic structure. We then evaluated their detection of changes in rhythmic structure in the context of randomly presented variations in tempo (rate) and frequency. Infants successfully differentiated between three-tones sequences with 1, 2 (X XX) and 2, 1 (XX X) structure as well as four-tone sequences with 2, 2 (XX XX) and 3, 1 (XXX X) structure. In other tasks, they indicated their ability to discriminate between contrasting tempos in the context of frequency variations. We conclude, then, that infants can categorize auditory sequences on the basis of rhythm and also on the basis of tempo.     

Spatial and temporal processing in the auditory and visual modalities

Three experiments tested the idea that auditory presentation facilitates temporal recall whereas spatial recall is better if the input modality is visual. Lists of words were presented in which the temporal and spatial orders were independent, and instructions to the subjects determined whether recall would be given in a spatial or temporal order. In all three experiments, a significant interaction between the input modality and the type of recall was found, such that visual presentation resulted in superior recall over auditory presentation in the spatial conditions and auditory presentation yielded superior recall to visual in the temporal conditions. The present results contradict an earlier study by Murdock that showed that auditory presentation resulted in better performance than visual presentation in a nominally spatial task. An explanation for the discrepancies between the results of that study and the present one is presented.     

Identification of temporal coherence between auditory and visual channels

Attention, Perception, & Psychophysics - The accuracy of identification of the temporal coherence between auditory and visual channels was examined. Random sequences of sound and light bursts...     

The temporal course of visual pattern encoding: Effects of pattern goodness

Subjects typically display superior reproduction of good (redundant, symmetrical) visual patterns compared with poor ones. This pattern goodness effect could conceivably involve encoding processes, short-term memory processes, or response processes. The present experiments explored the time course of wholistic encoding of Garner dot patterns as a function of tachistoscopic exposure time, delay of backward masking, and post-mask shadowing. Within the specific framework of additive factors theory, Experiment I showed: (a) equal rates of encoding for all patterns since comparable slopes were obtained for the recall X processing time functions; and (b) superior absolute recall for good patterns since different intercepts were obtained. Experiment II demonstrated that when degree of encoding was initially equalized for all patterns, the rate of extraction of further information remained constant over available processing time and was unaffected by pattern goodness, slopes and intercepts for good versus poor patterns then being equal. Experiment III confirmed that, given some fixed duration of available processing time, information is abstracted at the same rate for all pattern regardless of the ratio stimulus display time to delay of mask onset. Experiment IV indicated that maintenance rehearsal normally occurs in the present experimental situation, and that very good patterns are somewhat less disrupted by shadowing over a three-second interval. While STM is thus implicated in the pattern goodness effect it does not follow that STM constitutes a complete explanation of the intercept differences reported here. Empirical evidence of response bias toward production of good patterns, however, was not found. It was shown that very good patterns are highly familiar and nameable, and proposed that they do consequently have an early encoding advantage.     

Multiple temporal references in sensorimotor synchronization with metrical auditory sequences

A local phase perturbation in an auditory sequence during synchronized finger tapping elicits an automatic phase correction response (PCR). The stimulus for the PCR is usually considered to be the most recent tap-tone asynchrony. In this study, participants tapped on target tones ("beats") of isochronous tone sequences consisting of beats and subdivisions (1:n tapping). A phase perturbation was introduced either on a beat or on a subdivision. Both types of perturbation elicited a PCR, even though there was no asynchrony associated with a subdivision. Moreover, the PCR to a perturbed beat was smaller when an unperturbed subdivision followed than when there was no subdivision. The relative size of the PCRs to perturbed beats and subdivisions depended on tempo, on whether the subdivision was local or present throughout the sequence, and on whether or not participants engaged in mental subdivision, but not on whether or not taps were made on the subdivision level. The results show that phase correction in synchronization depends not merely on asynchronies but on perceptual monitoring of multiple temporal references within a metrical hierarchy.     

Statistical learning of higher-order temporal structure from visual shape sequences

In 3 experiments, the authors investigated the ability of observers to extract the probabilities of successive shape co-occurrences during passive viewing. Participants became sensitive to several temporal-order statistics, both rapidly and with no overt task or explicit instructions. Sequences of shapes presented during familiarization were distinguished from novel sequences of familiar shapes, as well as from shape sequences that were seen during familiarization but less frequently than other shape sequences, demonstrating at least the extraction of joint probabilities of 2 consecutive shapes. When joint probabilities did not differ, another higher-order statistic (conditional probability) was automatically computed, thereby allowing participants to predict the temporal order of shapes. Results of a single-shape test documented that lower-order statistics were retained during the extraction of higher-order statistics. These results suggest that observers automatically extract multiple statistics of temporal events that are suitable for efficient associative learning of new temporal features.     

Intramodal and crossmodal sensory transfer of visual and auditory temporal patterns

Temporal pattern perception for vision and audition was measured using pair comparisons of precisely determined rhythms as test items. Visual rhythms were more difficult to match than were comparable auditory ones. Crossmodal transfer equaled intramodal transfer in one experiment but was inferior in another. The differences between intra- and crossmodal transfer were related to the frequency of pattern elements, with crossmodal performance decreasing more when frequency rose. The nine test items showed a consistency in relative difficulty across experimental conditions. Differences in item difficulty were related to the symmetry of the first pattern in the item.     

Modality-constrained statistical learning of tactile, visual, and auditory sequences

The authors investigated the extent to which touch, vision, and audition mediate the processing of statistical regularities within sequential input. Few researchers have conducted rigorous comparisons across sensory modalities; in particular, the sense of touch has been virtually ignored. The current data reveal not only commonalities but also modality constraints affecting statistical learning across the senses. To be specific, the authors found that the auditory modality displayed a quantitative learning advantage compared with vision and touch. In addition, they discovered qualitative learning biases among the senses: Primarily, audition afforded better learning for the final part of input sequences. These findings are discussed in terms of whether statistical learning is likely to consist of a single, unitary mechanism or multiple, modality-constrained ones.     

Picking up the pace: effects of global temporal context on sensitivity to the tempo of auditory sequences

This article continues a line of research examining factors affecting listeners' auditory tempo sensitivities. Of particular interest is the question of whether listeners are sensitive to the overall (global) pace of their auditory environment and how this sensitivity may affect their perceptions of sequence timing. To address this question, we manipulated the set of sequence tempi (between 300 and 700 msec) that listeners experienced over the course of a 1-h period (i.e., the global temporal context) while they performed a tempo-discrimination task involving standard-comparison pairs of isochronous tone sequences. Overall findings show systematic distortions in perceived tempo that are consistent with the view that listeners adapt to the global pace of their auditory environments. Moreover, general support was found for the hypothesis that increasing the number of equal intervals in a standard sequence produces greater improvements in tempo sensitivity when the standard sequence tempo is different from the global pace than when it is at the global pace. Implications of these findings for models of timing and temporal processing are discussed.     

The perception of brief temporal intervals: power functions for auditory and visual stimulus intervals

Two experiments were performed to examine the role of method of estimation and the employment of a standard stimulus on the judged duration of auditory and visual stimuli presented for brief temporal intervals (0.25 to 5.0 s). The results indicate that the relationship between judged and physical duration is nearly direct and linear. Psychophysical methodology and stimulus modality exerted little influence on the obtained power functions.     

Psychological distance and emotional experience: what you see is what you get

Recent research suggests that perceiving negative emotion-eliciting scenes approaching intensifies the associated felt emotion, while perceiving emotion-eliciting scenes receding weakens the associated felt emotion (Muhlberger, Neumann, Wieser, & Pauli, 2008). In the present studies, we sought to extend these findings by examining the effects of imagining rather than perceiving such changes to negative emotion-eliciting scenes. Across three studies, we found that negative scenes generally elicited less negative responses and lower levels of arousal when imagined moving away from participants and shrinking, and more negative responses and higher levels of arousal when imagined moving toward participants and growing, as compared to the responses elicited by negative scenes when imagined unchanged. Patterns in responses to neutral scenes undergoing the same imagined transformations were similar on ratings of emotional arousal, but differed on valence-generally eliciting greater positivity when imagined moving toward participants and growing, and less positivity when imagined moving away from participants and shrinking. Moreover, for these effects to emerge, participants reported it necessary to explicitly imagine scenes moving closer or farther. These findings have implications for emotion regulation, and suggest that imagined spatial distance plays a role in mental representations of emotionally salient events.