Control of Expressive and Metronomic Timing in Pianists

In 12 tasks, each including 10 repetitions, 6 skilled pianists performed or responded to a musical excerpt. In the first 6 tasks, expressive timing was required; in the last 6 tasks, metronomic timing. The pianists first played the music on a digital piano (Tasks 1 and 7), then played it without auditory feedback (Tasks 2 and 8), then tapped on a response key in synchrony with one of their own performances (Tasks 3 and 9), with an imagined performance (Tasks 4 and 10), with a computer-generated performance (Tasks 5 and 11), and with a computer-generated sequence of clicks (Tasks 6 and 12). The results demonstrated that pianists are capable of generating the expressive timing pattern of their performance in the absence of auditory and kinaesthetic (piano keyboard) feedback. They can also synchronize their finger taps quite well with expressively timed music or clicks (while imagining the music), although they tend to underestimate long interonset intervals and to compensate on the following tap. Expressive timing is thus shown to be generated from an internal representation of the music. In metronomic performance, residual expressive timing effects were evident. Those did not depend on auditory feedback, but they were much reduced or absent when kinaesthetic feedback from the piano keyboard was eliminated. Thus, they seemed to arise from the pianist's physical interaction with the instrument. Systematic timing patterns related to expressive timing were also observed in synchronization with a metronomic computer performance and even in synchronization with metronomic clicks. These results shed light on intentional and unintentional, structurally governed processes of timing control in music performance.     

Timing in solo and duet piano performances

Perhaps some of the most refined forms of timing arise in musical performance, particularly in the coordination between musicians playing together. Studies of timing in solo and duet piano performances are described, in which the musicians gave repeat performances of the music. In both solo and duet performances there was expressive use of timing, modulating the tempo of the music and the phase relationship between the voices, and the expressive forms were similar in successive performances of the piece. There was also evidence of separate timing control of the metre and of the production of notes and rests. Thus timing in musical performance is best modelled by assuming two levels of timekeeper, one pacing the metre and the other contained in the movement trajectories of note production, computed by motor procedure in relation to the metre. It is argued that expressive forms are derived from an interpretation of the music rather than memorized; and that coordination between voices in the music is achieved at the level of the metre.     

Subjective visibility depends on level of processing

Is visual awareness graded or binary? Experimental work has provided support for both possibilities, leading to two coexisting but contradictory theoretical accounts. Here we propose a promising candidate factor through which to integrate both accounts: the depth of stimulus processing required by the task. We compared color identification (a low-level task) with numerical judgements (a high-level task) performed on the very same colored number stimuli. Psychophysical curves were analyzed for both objective discrimination performance and subjective visibility ratings on a trial-by trial basis. We observed a graded relationship between stimulus duration and visibility in the low-level task, but a more non-linear relationship in the high-level task. Both patterns of results have previously been consistently associated with the graded and the dichotomous account, respectively. Follow-up experiments that manipulate the level of processing can further unify previously inconsistent results, thus integrating two major theories of visual awareness.     

Perception of emotional expression in musical performance

Expression in musical performance is largely communicated by the manner in which a piece is played; interpretive aspects that supplement the written score. In piano performance, timing and amplitude are the principal parameters the performer can vary. We examined the way in which such variation serves to communicate emotion by manipulating timing and amplitude in performances of classical piano pieces. Over three experiments, listeners rated the emotional expressivity of performances and their manipulated versions. In Experiments 1 and 2, timing and amplitude information were covaried; judgments were monotonically decreasing with performance variability, demonstrating that the rank ordering of acoustical manipulations was captured by participants' responses. Further, participants' judgments formed an S-shaped (sigmoidal) function in which greater sensitivity was seen for musical manipulations in the middle of the range than at the extremes. In Experiment 3, timing and amplitude were manipulated independently; timing variation was found to provide more expressive information than did amplitude. Across all three experiments, listeners demonstrated sensitivity to the expressive cues we manipulated, with sensitivity increasing as a function of musical experience.     

Pitch direction ability predicts melodic perception in autism

Individuals with autism spectrum disorders (ASDs) often present atypical auditory perception. Previous work has reported both enhanced low-level pitch discrimination and superior abilities to detect local pitch structure on higher-level melodic tasks in ASD. However, it is unclear how low and high levels of auditory perception are related in ASD or typical development (TD), or how this relationship might change across development and stimulus presentation rates. To these aims, in the present study, children with ASD and TD were tested on a low-level pitch direction discrimination task and a high-level melodic global-local task. Groups performed similarly on both of these auditory tasks. Moreover, individual differences in low-level pitch direction ability predicted performance on the higher-level global-local task, with a stronger relationship in ASD. Age did not affect the relationship between low-level and high-level pitch performance in either ASD or TD. However, there was a more positive effect of age on the high-level global-local task performance in TD than ASD. Finally, there was no effect of stimulus rate on the relationship between low-level and high-level pitch performance in either group. These findings provide a better understanding of how perception is associated across levels of processing in ASD versus TD. This work helps to better understand individual differences in auditory perception and to refine ASD phenotypes.     

Visual dominance in the cross-modal kinesthetic to kinesthetic plus visual feedback condition

In previous studies subjects who have learned a positioning response with kinesthetic feedback tended to make greater errors when visual feedback was provided during later trials. These subjects have always performed with both kinesthetic and visual feedback available. This study determined whether subjects with only visual feedback would produce errors similar to those who received kinesthetic plus visual feedback. Blindfolded subjects learned to move a handle to a criterion location with knowledge of results following each trial. Subjects then were assigned to one of three experimental groups, with only kinesthetic feedback, with kinesthetic plus visual feedback, or with only visual feedback. Subjects had 9 trials without knowledge of results in these feedback conditions. When visual feedback was available, subjects tended to make longer response errors. This finding replicates previous studies. Also, the similarity of performances from the conditions with visual feedback indicated the dominance of visual information in the condition with kinesthetic plus visual feedback.     

Mapping musical thought to musical performance

Expressive timing methods are described that map pianists' musical thoughts to sounded performance. In Experiment 1, 6 pianists performed the same musical excerpt on a computer-monitored keyboard. Each performance contained 3 expressive timing patterns: chord asynchronies, rubato patterns, and overlaps (staccato and legato). Each pattern was strongest in experienced pianists' performances and decreased when pianists attempted to play unmusically. In Experiment 2 pianists performed another musical excerpt and notated their musical intentions on an unedited score. The notated interpretations correlated with the presence of the 3 methods: The notated melody preceded other events in chords (chord asynchrony); events notated as phase boundaries showed greatest tempo changes (rubato); and the notated melody showed most consistent amount of overlap between adjacent events (staccato and legato). These results suggest that the mapping of musical thought to musical action is rule-governed, and the same rules produce different interpretations.     

Fake feedback on pain tolerance impacts proactive versus reactive control strategies

It is well-known that beliefs about one’s own ability to execute a task influence task performance. Here, we tested the hypothesis that beliefs about a specific self-control capacity, namely pain tolerance, modulate basic cognitive control processes. Participants received fake comparative social feedback that their ability to tolerate painful stimulations was either very poor or outstanding after which they performed an unrelated go/no-go task. Participants receiving low-tolerance feedback, relative to high-tolerance feedback, were less successful at inhibiting their responses and more influenced by previous trial conditions, as indicated by an increased slowdown following errors and more failed inhibitions following go-trials. These observations demonstrate a shift from a more proactive to a more reactive control mode. This study shows that providing feedback about one’s own capacity to control impulsive reactions to painful stimulations directly influences low-level cognitive control dynamics.     

Detectability of duration and intensity increments in melody tones: A partial connection between music perception and performance

Two experiments demonstrate positional variation in the relative detectability of, respectively, local temporal and dynamic perturbations in an isochronous and isodynamic sequence of melody tones, played on a computer-controlled piano. This variation may reflect listeners’ expectations of expressive performance microstructure (thetop-down hypothesis), or it may be due to psychoacoustic (pitch-related) stimulus factors (thebottom-up hypothesis). Percent correct scores for increments in tone duration correlated significantly with the average timing profile of pianists’ expressive performances of the music, as predicted specifically by the top-down hypothesis. For intensity increments, the analogous perception-performance correlation was weak and the bottom-up factors of relative pitch height and/or direction of pitch change accounted for some of the perceptual variation. Subjects’ musical training increased overall detection accuracy but did not affect the positional variation in accuracy scores in either experiment. These results are consistent with the top-down hypothesis for timing, but they favor the bottom-up hypothesis for dynamics. The perception-performance correlation for timing may also be viewed as being due to complex stimulus properties such as tonal motion and tension/relaxation that influence performers and listeners in similar ways.     

Variability of timing in expressive piano performance increases with interval duration

In simple motor tasks such as finger tapping at different constant rates, within-trial variability of response interonset intervals (IOIs) increases with IOI duration (which varies between trials). In expressive piano performance, the rate of key depressions is not constant, in part due to compositional structure and in part due to expressive timing, so that IOIs of many different durations occur within a single “trial.” Nevertheless, across repeated performances of the same music (Schumann’s “Träumerei” and Debussy’s “La fille aux cheveux de lin”) at the same intended tempo, the standard deviations of individual IOIs tend to increase linearly with their average duration. This is also true when the variation is due to expressive timing alone and when unintended differences in basic tempo between performances are taken into account. In the music studied here, at least, there was no evidence of compensatory timing. The results suggest that the pianists employed a continuously variable tempo governed by a flexible internal timekeeper whose variability follows a generalized Weber’s law (for IOIs longer than about 300 msec).     

Abstract and concrete self-evaluative goals

Assuming that people often hold the abstract goal of acquiring accurate feedback but recognize that acquiring favorable feedback can make the self-evaluative process more comfortable, the authors posited that low-level construals (of how action is performed) would elicit greater self-enhancement motivation than would high-level construals (of why action is performed). Individuals chronically using low-level construals had greater interest in downward social comparison (DSC) and less interest in negative feedback (NF; Studies 1 and 3). Decreases in temporal distance (which foster low-level construals) also elicited greater interest in DSC and less interest in NF (Studies 2 and 4). The latter effect was explained by participants' aversion to inconvenience (Study 5) and not by approach-avoidance conflict (Study 6). These results suggest that the level of abstraction at which people construe self-evaluative situations can influence their feedback preferences.     

Self-induced versus reactive triggering of synchronous movements in a deafferented patient and control subjects

The present study investigates the contribution of tactile-kinesthetic information to the timing of movements. The relative timing of simultaneous tapping movements of finger and foot (hand-foot asynchrony) was examined in a simple reaction time task and in discrete self-initiated taps (Experiment 1), and in externally triggered synchronization tapping (Experiment 2). We compared the performance of a deafferented participant (IW) to the performance of two control groups of different ages. The pattern of results in control groups replicates previous findings: Whereas positive hand-foot asynchronies (hand precedes foot) are observed in a simultaneous reaction to an auditory stimulus, hand-foot asynchronies are negative with discrete self-initiated as well as auditorily paced sequences of synchronized finger and foot taps. In the first case, results are explained by a simultaneous triggering of motor commands. In contrast, self-initiated and auditorily paced movements are assumed to be controlled in terms of their afferent consequences, as provided by tactile-kinesthetic information. The performance of the deafferented participant differed from that of healthy participants in some aspects. As expected on the basis of unaffected motor functions, the participant was able to generate finger and foot movements in reaction to an external signal. In spite of the lack of movement-contingent sensory feedback, the deafferented participant showed comparable timing errors in self-initiated and regularly paced tapping as observed in control participants. However, in discrete self-initiated taps IW's hand-foot asynchronies were considerably larger than in control participants, while performance did not differ from that of controls in continuous movement generation. These findings are discussed in terms of an internal generation of the movement's sensory consequences (forward-modeling).     

The levels of perceptual processing and the neural correlates of increasing subjective visibility

According to the levels-of-processing hypothesis, transitions from unconscious to conscious perception may depend on stimulus processing level, with more gradual changes for low-level stimuli and more dichotomous changes for high-level stimuli. In an event-related fMRI study we explored this hypothesis using a visual backward masking procedure. Task requirements manipulated level of processing. Participants reported the magnitude of the target digit in the high-level task, its color in the low-level task, and rated subjective visibility of stimuli using the Perceptual Awareness Scale. Intermediate stimulus visibility was reported more frequently in the low-level task, confirming prior behavioral results. Visible targets recruited insulo-fronto-parietal regions in both tasks. Task effects were observed in visual areas, with higher activity in the low-level task across all visibility levels. Thus, the influence of level of processing on conscious perception may be mediated by attentional modulation of activity in regions representing features of consciously experienced stimuli.     

不同类型反馈对儿童学习效果的影响及性别差异

儿童阶段是对外部反馈最敏感的阶段。以往研究发现,不同类型反馈（积极/消极反馈）和不同性质的强化物（物质/社会性强化）对儿童学习效果存在交互作用,并且对不同性别儿童的影响有所不同。本研究通过两个实验,采用联结学习范式以考察不同类型反馈对8~10岁儿童学习效果的影响。首先考察了积极反馈和消极反馈对儿童学习效果的影响,进而在积极、消极反馈的基础上加入了物质、社会性强化物,探究物质、社会性强化条件下积极、消极反馈对儿童学习效果影响的性别差异。结果表明,对于8~10岁儿童来说,消极反馈比积极反馈对儿童学习效果的影响更大。并且,物质、社会性强化物对儿童反馈学习效果的影响存在性别差异,对于男孩来说,在物质性强化条件下,消极反馈更能促进其学习;而对于女孩来说,在社会性强化物下,消极反馈更能促进她们的学习。该研究为如何运用反馈促进儿童学习提供了实证依据。     

Self-denial and the role of intentions in the attribution of agency

The ability to distinguish between our own actions and those of an external agent is a fundamental component of normal human social interaction. Both low- and high-level mechanisms are thought to contribute to the sense of movement agency, but the contribution of each is yet to be fully understood. By applying small and incremental perturbations to realistic visual feedback of the limb, the influence of high-level action intentions and low-level motor predictive mechanisms were dissociated in two experiments. In the first, participants were induced to claim agency over movements that were subject to large perturbations and to deny agency over self-produced unperturbed movements despite the application of motor corrections by low-level mechanisms. A control experiment confirmed that if reaches met with their intended goal then they were more likely to be attributed to the agent, regardless of the discrepancy between the actual and seen positions of the limb.     

Seeing the light: Adapting luminance reveals low-level visual processes in the attentional blink

It is widely assumed that high-level visual processes subserve the attentional blink (AB). Recent evidence from studies of visual masking during the AB that were designed to directly test the contributions of high-level masking effects, however, have failed to provide empirical support for this position. The implication is that low-level visual processes are crucial to the AB. We tested this idea by manipulating adapting luminance in a standard AB paradigm. Consistent with the involvement of low-level neural mechanisms, the AB effect interacted with adapting luminance such that an AB was revealed only under photopic (light adapted) viewing conditions.     

Crossmodal interactions in the perception of expressivity in musical performance

In musical performance, bodily gestures play an important role in communicating expressive intentions to audiences. Although previous studies have demonstrated that visual information can have an effect on the perceived expressivity of musical performances, the investigation of audiovisual interactions has been held back by the technical difficulties associated with the generation of controlled, mismatching stimuli. With the present study, we aimed to address this issue by utilizing a novel method in order to generate controlled, balanced stimuli that comprised both matching and mismatching bimodal combinations of different expressive intentions. The aim of Experiment 1 was to investigate the relative contributions of auditory and visual kinematic cues in the perceived expressivity of piano performances, and in Experiment 2 we explored possible crossmodal interactions in the perception of auditory and visual expressivity. The results revealed that although both auditory and visual kinematic cues contribute significantly to the perception of overall expressivity, the effect of visual kinematic cues appears to be somewhat stronger. These results also provide preliminary evidence of crossmodal interactions in the perception of auditory and visual expressivity. In certain performance conditions, visual cues had an effect on the ratings of auditory expressivity, and auditory cues had a small effect on the ratings of visual expressivity.     

Goal-directed aiming under restricted viewing conditions with confirmatory sensory feedback

A substantial body of research has examined the speed-accuracy tradeoff captured by Fitts’ law, demonstrating increases in movement time that occur as aiming tasks are made more difficult by decreasing target width and/or increasing the distance between targets. Yet, serial aiming movements guided by internal spatial representations, rather than by visual views of targets have not been examined in this manner, and the value of confirmatory feedback via different sensory modalities within this paradigm is unknown. Here we examined goal-directed serial aiming movements (tapping back and forth between two targets), wherein targets were visually unavailable during the task. However, confirmatory feedback (auditory, haptic, visual, and bimodal combinations of each) was delivered upon each target acquisition, in a counterbalanced, within-subjects design. Each participant performed the aiming task with their pointer finger, represented within an immersive virtual environment as a 1 cm white sphere, while wearing a head-mounted display. Despite visual target occlusion, movement times increased in accordance with Fitts’ law. Though Fitts’ law captured performance for each of the sensory feedback conditions, the slopes differed. The effect of increasing difficulty on movement times was least influential in the haptic condition, suggesting more efficient processing of confirmatory haptic feedback during aiming movements guided by internal spatial representations.     

Reliance on visual feedback from ocular accommodation on motor skills in children with developmental coordination disorder and typically developing controls

Children with developmental coordination disorder (DCD) present with marked impairments in motor skills, including visual-motor integration. Oculomotor anomalies are more prevalent in children with DCD than typically developing children. Children with DCD further demonstrate altered use of visual feedback compared to typically developing controls. We investigated whether the accommodation system, a key component of the oculomotor system, contributes to visual feedback during fine and gross motor skills performance; and whether children with DCD demonstrate differences in reliance on visual feedback from accommodation. Minus dioptre lenses were used to maximally induce accommodation and impede accommodation dynamics. Children with DCD and typically developing controls performed motor skills tests assessing balance, upper limb coordination, visual-motor performance, gross and fine dexterity. Motor skills performance in controls was significantly affected by impeded accommodation in all tasks. Children with DCD demonstrated reliance on accommodation feedback in upper limb and visual-motor tasks only. Children with DCD may be less reliant on visual feedback obtained from accommodation due to adaptive mechanisms to overcome faulty information in the presence of oculomotor anomalies. These results strengthen our previous findings that accommodation anomalies contribute to motor skills impairment, and suggest that performance on these motor tasks is heavily reliant on visual feedback from accommodation.