Control of Expressive and Metronomic Timing in Pianists

In 12 tasks, each including 10 repetitions, 6 skilled pianists performed or responded to a musical excerpt. In the first 6 tasks, expressive timing was required; in the last 6 tasks, metronomic timing. The pianists first played the music on a digital piano (Tasks 1 and 7), then played it without auditory feedback (Tasks 2 and 8), then tapped on a response key in synchrony with one of their own performances (Tasks 3 and 9), with an imagined performance (Tasks 4 and 10), with a computer-generated performance (Tasks 5 and 11), and with a computer-generated sequence of clicks (Tasks 6 and 12). The results demonstrated that pianists are capable of generating the expressive timing pattern of their performance in the absence of auditory and kinaesthetic (piano keyboard) feedback. They can also synchronize their finger taps quite well with expressively timed music or clicks (while imagining the music), although they tend to underestimate long interonset intervals and to compensate on the following tap. Expressive timing is thus shown to be generated from an internal representation of the music. In metronomic performance, residual expressive timing effects were evident. Those did not depend on auditory feedback, but they were much reduced or absent when kinaesthetic feedback from the piano keyboard was eliminated. Thus, they seemed to arise from the pianist's physical interaction with the instrument. Systematic timing patterns related to expressive timing were also observed in synchronization with a metronomic computer performance and even in synchronization with metronomic clicks. These results shed light on intentional and unintentional, structurally governed processes of timing control in music performance.     

原因调节与反应调节的情绪变化过程

采用生理心理实验法研究忽视、抑制、重视、宣泄等方式调节负情绪的情绪变化过程,发现忽视有效减弱了主观感受和表情行为,并引起R-R问期更大的增加;抑制不能减弱主观感受,并引起手指脉搏血容振幅更大的增幅;重视增强了主观感受;宣泄增强了主观感受,并相对地减弱了生理激活水平。研究表明原因调节可以更有效地调整负情绪主观感受,反应调节使负情绪成分变化出现“水压模型”式循环动力特点;情绪调节过程实现着情绪在心理适应中的促进(或阻碍)作用。     

Simulating a skilled typist: a study of skilled cognitive-motor performance

We review the major phenomena of skilled typing and propose a model for the control of the hands and fingers during typing. The model is based upon an Activation-Trigger-Schema system in which a hierarchical structure of schemata directs the selection of the letters to be typed and, then, controls the hand and finger movements by a cooperative, relaxation algorithm. The interactions of the patterns of activation and inhibition among the schemata determine the temporal ordering for launching the keystrokes. To account for the phenomena of doubling errors, the model has only “type” schemata—no “token” schemata—with only a weak binding between the special schema that signals a doubling, and its argument. The model exists as a working computer simulation and produces an output display of the hands and fingers moving over the keyboard. It reproduces some of the major phenomena of typing, including the interkeystroke interval times, the pattern of transposition errors found in skilled typists, and doubling errors. Although the model is clearly inadequate or wrong in some of its features and assumptions, it serves as a useful first approximation for the understanding of skilled typing.     

Variability of timing in expressive piano performance increases with interval duration

In simple motor tasks such as finger tapping at different constant rates, within-trial variability of response interonset intervals (IOIs) increases with IOI duration (which varies between trials). In expressive piano performance, the rate of key depressions is not constant, in part due to compositional structure and in part due to expressive timing, so that IOIs of many different durations occur within a single “trial.” Nevertheless, across repeated performances of the same music (Schumann’s “Träumerei” and Debussy’s “La fille aux cheveux de lin”) at the same intended tempo, the standard deviations of individual IOIs tend to increase linearly with their average duration. This is also true when the variation is due to expressive timing alone and when unintended differences in basic tempo between performances are taken into account. In the music studied here, at least, there was no evidence of compensatory timing. The results suggest that the pianists employed a continuously variable tempo governed by a flexible internal timekeeper whose variability follows a generalized Weber’s law (for IOIs longer than about 300 msec).     

Perceiving action identity: how pianists recognize their own performances

Can skilled performers, such as artists or athletes, recognize the products of their own actions? We recorded 12 pianists playing 12 mostly unfamiliar musical excerpts, half of them on a silent keyboard. Several months later, we played these performances back and asked the pianists to use a 5-point scale to rate whether they thought they were the person playing each excerpt (1 = no, 5 = yes). They gave their own performances significantly higher ratings than any other pianist's performances. In two later follow-up tests, we presented edited performances from which differences in tempo, overall dynamic (i.e., intensity) level, and dynamic nuances had been removed. The pianists' ratings did not change significantly, which suggests that the remaining information (expressive timing and articulation) was sufficient for self-recognition. Absence of sound during recording had no significant effect. These results are best explained by the hypothesis that an observer's action system is most strongly activated during perception of self-produced actions.     

A Dual Process Account of Coarticulation in Motor Skill Acquisition

Many tasks, such as typing a password, are decomposed into a sequence of subtasks that can be accomplished in many ways. Behavior that accomplishes subtasks in ways that are influenced by the overall task is often described as “skilled” and exhibits coarticulation. Many accounts of coarticulation use search methods that are informed by representations of objectives that define skilled. While they aid in describing the strategies the nervous system may follow, they are computationally complex and may be difficult to attribute to brain structures. Here, the authors present a biologically- inspired account whereby skilled behavior is developed through 2 simple processes: (a) a corrective process that ensures that each subtask is accomplished, but does not do so skillfully and (b) a reinforcement learning process that finds better movements using trial and error search that is not informed by representations of any objectives. We implement our account as a computational model controlling a simulated two-armed kinematic “robot” that must hit a sequence of goals with its hands. Behavior displays coarticulation in terms of which hand was chosen, how the corresponding arm was used, and how the other arm was used, suggesting that the account can participate in the development of skilled behavior.     

A minicomputer program for control and data acquisition in classical conditioning

A 4K PDP-8/E computer program package has been developed to control classical conditioning procedures, to collect response data, and to extract statistical summeries. In contrast with other existing behavioral control languages geared to digital data typical of discrete operant analyses, this program distinguishes itself by its ability to retrieve and analyze the analog data arising from phasic response systems such as the rabbit’s nictitating membrane. By means of a question-and-answer Teletype conversation with the E, the program establishes trial sequence and trial spacing parameters; CS and US duration, probability, and interstimulus interval; intertrial interval fractionation for recording intertrial response frequencies; and session length. Various versions of the program exist to compute statistical properties of the analog response data, to dump detailed trial-by-trial topographies, and to attach instrumental contingencies to subtle features of the real-time analog responding.     

Category discrimination by pigeons using five polymorphous features

Eight pigeons were trained to discriminate between sets of color photographs of natural scenes. The scenes differed along five two-valued dimensions (site, weather, camera distance, camera orientation, and camera height), and all combinations of the feature values were used. One value of each dimension was designated as positive, and slides containing three or more positive feature values were members of the positive stimulus set. Thus, each feature had an equal, low, correlation with reinforcement, and all features had zero correlations with each other. Seven of the 8 pigeons learned this discrimination, and their responding came under the control of all five features. Within the positive and negative stimulus sets, response rates were higher to stimuli that contained more positive feature values. Once discrimination had been achieved, reversal training was given using a subset of the slides. In this subset, only a single feature was correlated with reinforcement. All pigeons learned this reversal successfully and generalized it to additional photographs with the same feature content. After reversal, the original reinforcement contingencies were reinstated, and training was continued using all the slides except those that had been used in reversal. Reversal generalized to these slides to some extent. Analysis of the response rates to individual slides showed that, compared with prereversal training, only the feature that had been subjected to reversal contingencies showed a reversed correlation with response rate. The remaining features showed the same correlation with response rate as they had before reversal training. Thus, reversal on some members of a category following category discrimination training led to generalization to stimuli within the category that were not involved in the reversal, but not to features that were not reversed. It is therefore inappropriate to refer to the pigeons as learning a concept.     

Identifying the mechanisms underpinning recognition of structured sequences of action

We present three experiments to identify the specific information sources that skilled participants use to make recognition judgements when presented with dynamic, structured stimuli. A group of less skilled participants acted as controls. In all experiments, participants were presented with filmed stimuli containing structured action sequences. In a subsequent recognition phase, participants were presented with new and previously seen stimuli and were required to make judgements as to whether or not each sequence had been presented earlier (or were edited versions of earlier sequences). In Experiment 1, skilled participants demonstrated superior sensitivity in recognition when viewing dynamic clips compared with static images and clips where the frames were presented in a nonsequential, randomized manner, implicating the importance of motion information when identifying familiar or unfamiliar sequences. In Experiment 2, we presented normal and mirror-reversed sequences in order to distort access to absolute motion information. Skilled participants demonstrated superior recognition sensitivity, but no significant differences were observed across viewing conditions, leading to the suggestion that skilled participants are more likely to extract relative rather than absolute motion when making such judgements. In Experiment 3, we manipulated relative motion information by occluding several display features for the duration of each film sequence. A significant decrement in performance was reported when centrally located features were occluded compared to those located in more peripheral positions. Findings indicate that skilled participants are particularly sensitive to relative motion information when attempting to identify familiarity in dynamic, visual displays involving interaction between numerous features.     

Identifying the mechanisms underpinning recognition of structured sequences of action

We present three experiments to identify the specific information sources that skilled participants use to make recognition judgements when presented with dynamic, structured stimuli. A group of less skilled participants acted as controls. In all experiments, participants were presented with filmed stimuli containing structured action sequences. In a subsequent recognition phase, participants were presented with new and previously seen stimuli and were required to make judgements as to whether or not each sequence had been presented earlier (or were edited versions of earlier sequences). In Experiment 1, skilled participants demonstrated superior sensitivity in recognition when viewing dynamic clips compared with static images and clips where the frames were presented in a nonsequential, randomized manner, implicating the importance of motion information when identifying familiar or unfamiliar sequences. In Experiment 2, we presented normal and mirror-reversed sequences in order to distort access to absolute motion information. Skilled participants demonstrated superior recognition sensitivity, but no significant differences were observed across viewing conditions, leading to the suggestion that skilled participants are more likely to extract relative rather than absolute motion when making such judgements. In Experiment 3, we manipulated relative motion information by occluding several display features for the duration of each film sequence. A significant decrement in performance was reported when centrally located features were occluded compared to those located in more peripheral positions. Findings indicate that skilled participants are particularly sensitive to relative motion information when attempting to identify familiarity in dynamic, visual displays involving interaction between numerous features.     

Controlling relations in conditional discrimination and matching by exclusion.

Normally capable adults learned two-choice identity matching of three-digit numerals and arbitrary matching of physically dissimilar nonsense syllables. The stimuli were displayed on a computer terminal, and responses consisted of typing on the terminal's keyboard. In Experiment 1, every trial displayed a sample numeral, a comparison numeral, and three equal signs (= = =). The comparison stimulus was to be selected if it was identical with the sample; otherwise the equal sign was to be selected. This "single comparison" method was then used to show that arbitrary matching could be based upon either sample-S+ or sample-S- relations. In Experiment 2, a series of probe trials displayed a novel sample, a comparison stimulus from the arbitrary matching baseline, and = = =. Subjects typically selected = = =; they apparently were excluding the baseline comparison stimulus. Experiments 3 through 5 investigated which variables in training would lead to the selection of baseline comparison stimuli in response to novel samples. Behavior was usually unchanged when baseline training included relating comparison stimuli to as many as four different samples. Punishment contingencies were effective, but performance did not generalize unless those contingencies were applied in relation to more than one baseline comparison stimulus.     

Response programming, as assessed by reaction time, does not establish commands for particular muscles

In previous studies, response programming has been inferred from a relation between reaction time and the nature of the response which follows. However, it has not been clear whether this programming process generates commands for specific muscles or abstract timing networks which can be applied to any appropriate muscle group. Experiment 1 employed the Sternberg (1969) additive-factor method to show that muscle selection need not be completed before such programming begins, a conclusion which is inconsistent with the view that this process establishes commands to previously selected muscles. Experiments 2 and 3 provide converging evidence against the muscle-specific view of programming by showing that advance programming of response timing can occur when the response muscle is not specified. A theoretical framework encompassing these and previous results is proposed.     

Perception and production of brief durations: beat-based versus interval-based timing

Two experiments examined whether timing of short intervals is beat- or interval-based. In Experiment 1, subjects heard a sequence of standard tones followed by 2 test tones; they compared the interval between test tones to the interval between the standards. If optimal precision required beat-based timing, performance should be best in blocks in which the interval between standard and test reliably matched the standard interval. No such effect was observed. In Experiment 2, subjects heard 2 test tones and reproduced the intertone interval by producing 2 keypress responses. Entrainment to the beat was apparent: First-response latency clustered around the standard interval and was positively correlated with the produced interval. However, responses occurring on or near the beat showed no better temporal fidelity than off-beat responses. One plausible interpretation of these findings is that the brain always times brief intervals with an interval timer; however, this timer can be used in a cyclic fashion to trigger rhythmic responses.     

Resurgence of integrated behavioral units

Two experiments with rats examined the dynamics of well-learned response sequences when reinforcement contingencies were changed. Both experiments contained four phases, each of which reinforced a 2-response sequence of lever presses until responding was stable. The contingencies then were shifted to a new reinforced sequence until responding was again stable. Extinction-induced resurgence of previously reinforced, and then extinguished, heterogeneous response sequences was observed in all subjects in both experiments. These sequences were demonstrated to be integrated behavioral units, controlled by processes acting at the level of the entire sequence. Response-level processes were also simultaneously operative. Errors in sequence production were strongly influenced by the terminal, not the initial, response in the currently reinforced sequence, but not by the previously reinforced sequence. These studies demonstrate that sequence-level and response-level processes can operate simultaneously in integrated behavioral units. Resurgence and the development of integrated behavioral units may be dissociated; thus the observation of one does not necessarily imply the other.     

The perception of expressive timing in music

Summary This paper is concerned with the perception of small-scale timing changes in musical sequences. The control and expressive function of these have been studied quite extensively from a production perspective, but not much is known about listeners' ability to detect them. A pilot study and two experiments are reported which investigate the detectability of different amounts of timing change in different sequential positions, different pitch contexts, and against the background of both metronomic and expressive comparisons. The results show that listeners are able to perceive as little as 20 ms lengthening in the context of notes lasting between 100 and 400 ms, and that this threshold appears not to be a function of base duration in this range. Sequential position and pitch structure influence the detectability of timing changes to a limited extent, for which some possible explanations are offered. A case is made for regarding timing in music as both a medium to convey structure and an object in its own right, suggesting that it may be perceptually organized in two different ways — as the consequence of a structural interpretation and as a directly registered quantity.     

Perception of emotional expression in musical performance

Expression in musical performance is largely communicated by the manner in which a piece is played; interpretive aspects that supplement the written score. In piano performance, timing and amplitude are the principal parameters the performer can vary. We examined the way in which such variation serves to communicate emotion by manipulating timing and amplitude in performances of classical piano pieces. Over three experiments, listeners rated the emotional expressivity of performances and their manipulated versions. In Experiments 1 and 2, timing and amplitude information were covaried; judgments were monotonically decreasing with performance variability, demonstrating that the rank ordering of acoustical manipulations was captured by participants' responses. Further, participants' judgments formed an S-shaped (sigmoidal) function in which greater sensitivity was seen for musical manipulations in the middle of the range than at the extremes. In Experiment 3, timing and amplitude were manipulated independently; timing variation was found to provide more expressive information than did amplitude. Across all three experiments, listeners demonstrated sensitivity to the expressive cues we manipulated, with sensitivity increasing as a function of musical experience.     

Detectability of duration and intensity increments in melody tones: A partial connection between music perception and performance

Two experiments demonstrate positional variation in the relative detectability of, respectively, local temporal and dynamic perturbations in an isochronous and isodynamic sequence of melody tones, played on a computer-controlled piano. This variation may reflect listeners’ expectations of expressive performance microstructure (thetop-down hypothesis), or it may be due to psychoacoustic (pitch-related) stimulus factors (thebottom-up hypothesis). Percent correct scores for increments in tone duration correlated significantly with the average timing profile of pianists’ expressive performances of the music, as predicted specifically by the top-down hypothesis. For intensity increments, the analogous perception-performance correlation was weak and the bottom-up factors of relative pitch height and/or direction of pitch change accounted for some of the perceptual variation. Subjects’ musical training increased overall detection accuracy but did not affect the positional variation in accuracy scores in either experiment. These results are consistent with the top-down hypothesis for timing, but they favor the bottom-up hypothesis for dynamics. The perception-performance correlation for timing may also be viewed as being due to complex stimulus properties such as tonal motion and tension/relaxation that influence performers and listeners in similar ways.     

Learning expressive percussion performance under different visual feedback conditions

A study was conducted to test the effect of two different forms of real-time visual feedback on expressive percussion performance. Conservatory percussion students performed imitations of recorded teacher performances while receiving either high-level feedback on the expressive style of their performances, low-level feedback on the timing and dynamics of the performed notes, or no feedback. The high-level feedback was based on a Bayesian analysis of the performances, while the low-level feedback was based on the raw participant timing and dynamics data. Results indicated that neither form of feedback led to significantly smaller timing and dynamics errors. However, high-level feedback did lead to a higher proficiency in imitating the expressive style of the target performances, as indicated by a probabilistic measure of expressive style. We conclude that, while potentially disruptive to timing processes involved in music performance due to extraneous cognitive load, high-level visual feedback can improve participant imitations of expressive performance features.     

Operant control of response latency in monkeys: evidence for a central explanation

Two monkeys (M. mulatta) were trained to press a telegraph key after onset of a tone and release it quickly in response to a subsequent light or click stimulus occurring after a variable interval. After training first with a fixed time limit on response latency for key release and then with a continuously adjusting limit, reaction time to click was 160 msec and to light, about 200 msec. Temporal contingencies or "payoff bands" were then introduced which reinforced only responses with latencies which fell between two limits 50 msec apart. Feedback was given as to whether each latency was too slow, within the band, or too fast. A trained monkey could precisely center its latency distribution on any 50 msec-wide payoff band located from 200 to 600 msec after the stimulus, with from 60 to 80% of its responses achieving reinforcement. Distribution statistics were comparable to those of trained human subjects. Because such precise timing might be accomplished by a peripheral adjustment, such as changing the manner of holding the key, latency of electromyographic activation was measured in participating arm muscles in one monkey. Electromyographic activation preceded key release by a constant interval, regardless of response latency, indicating a more central mechanism for timing of brief intervals.