The non-occurrence of a stimulus as a signal

Two groups of rats were rewarded for pressing a panel following a varying number of bar presses; a signal following the bar press indicated whether or not a panel press would be rewarded if made before the next bar press. For one group the signal indicating reward was a flash of light, for the other it was the non-occurrence of the flash (in the latter case the flash thus signalled non-reward). The first group learned to withold panel presses except when reward was signalled, but the second group did not. This result is related to the “feature positive effect” of Jenkins and Sainsbury (1970), in which pigeons failed to withold pecks at a negative stimulus display when it was the same as the positive display except for the addition of a distinctive feature.     

Memory retrieval and discrimination learning

A better reinforcement schedule may be associated with one exteroceptive stimulus (S+) than with another exteroceptive stimulus (S−). While all theories agree that performance in such discrimination tasks is affected by the reinforcement schedules associated with the exteroceptive S+ and S− cues, some theories suggest that performance is also affected by the reinforcement schedule associated with interoceptive reward produced cues. The two experiments reported here were concerned with identifying the conditions under which the interoceptive cues produced by nonreward come to affect performance. According to one hypothesis, such cues will acquire control over approach responding if animals make an approach response in their presence, but not otherwise. According to the memory hypothesis, the memory of nonreinforcement will become a signal for reinforcement, thus invigorating performance, if it is retrieved on a rewarded trial. In two experiments, two groups made strong approach responses to the nonreinforced cues on S− trials, but the memory of nonreward was better retrieved on a subsequent rewarded trial in one group than in the other. Subsequently, both groups were extinguished. The extinction findings support the memory view.     

The differential reinforcement of reward-produced and response-produced stimuli

In two experiments, groups received successive large-reward trials on odd-numbered days and successive small-reward trials on even-numbered days in the same gray alleyway. This produced a discrimination problem in which the memory of large reward (S^L) was reliably discriminative of large reward and the memory of small reward (S^S) was reliably discriminative of small reward. Intertrial interval (ITI) was varied both between and within groups. In the within-groups manipulation, ITI separating S+ trials differed from that separating S- trials. Experimental groups learned the discrimination, running slower to S^S (S- cue) than to S^L (S+ cue), and showed a negative contrast effect, running slower to S- than a small-reward control group. Discrimination was somewhat faster at massed than at spaced trials. The within-groups manipulation of ITI suggested that the effects of ITI were mediated by time-dependent changes in internal cues produced by reward events and by the instrumental response. The control exercised by internal cues was shown to be associative rather than nonassociative, e.g., motivational. Similarities with, and implications for, conventional brightness differential conditioning were discussed.     

The timing of bar-pressing behaviour

The time spent by a rat in a bar-pressing situation is made up of active time spent n pressing, eating time, and extra time spent in other activities. With a well trained rat, active time and extra time are small, and eating time mainly determines the rate of reward delivery. Active time is affected by a change of weight on the bar, the time between reward deliveries is affected by the amount of reward, and the extra time is affected by extinction conditions.

There is not a one-to-one correspondence between periods of activity at the knob and rewards.

The term “response” and some variables based on it are given empirical referents, which show that much research and theorizing on bar-pressing behaviour has been concerned with only a small selection of the rat's bar-pressing activities. Some reasons for this restriction are the use of the simple weighted bar, the lack of a rationale for bar-pressing research, and the practice of not watching the rat during an experiment.     

The interaction of hunger and thirst in the rat

Two groups of rats were given a series of trials in an enclosed runway with a food reward at the end, one group being run hungry, the other hungry plus thirsty. Then each group was split into three sub-groups: one run hungry, the second thirsty and the other hungry plus thirsty, in each case without food reward.

It was found that, whereas on the rewarded runs the extra, “irrelevant,” thirst increased running speed, on unrewarded runs it had the opposite effect and slowed up performance. Thus on unrewarded runs the two sub-groups running thirsty, and hungry plus thirsty, ran as slowly as those running hungry. Differences were found not to depend on whether the animals had been hungry or hungry plus thirsty on previous rewarded runs.

The interaction of primary needs therefore depends on the external situation. This can be accounted for in terms of the Pavlovian theories of mutual induction and conditioning, but not in terms of Hull's theory of “irrelevant drives.”     

Unconscious reward facilitates motion perceptual learning

Reward signal plays an important role in guiding human learning behaviour. Recent studies have provided evidence that reward signal modulates perceptual learning of basic visual features. Typically, the reward effects on perceptual learning were accompanied with consciously presented reward during the learning process. However, whether an unconsciously presented reward signal that minimizes the contribution of attentional and motivational factors can facilitate perceptual learning remains less well understood. We trained human subjects on a visual motion detection task and subliminally delivered a monetary reward for correct response during the training. The results showed significantly larger learning effect for high reward-associated motion direction than low reward-associated motion direction. Importantly, subjects could neither discriminate the relative values of the subliminal monetary reward nor correctly report the reward-direction contingencies. Our findings suggest that reward signal plays an important modulatory role in perceptual learning even if the magnitude of the reward was not consciously perceived.     

Treatment of nocturnal enuresis by conditioning techniques

One hundred and fifteen enuretic children were assigned to five treatment groups. Three groups received Mowrer-type continuous signal (C.S.), Twin-Signal (T.S.) or intermittent Twin Signal (T.S.-I.R.) conditioning treatment, and two groups were given “placebo” or “arousal” control treatments. Over a one-month period, there was no difference between conditioning and control procedures, and it was concluded that further research is needed to determine the basis of treatment response. There was no evidence in support of the escape training hypothesis, and the twin-signal modification to standard bell-and-pad treatment is not recommended. The investigation confirmed the success of conditioning treatment in bringing about the initial arrest of enuresis (i.e. in 81.4 per cent of the cases) but the relapse rate was high. The investigation provided tentative evidence that intermittent conditioning treatment offers one way of reducing the frequency of relapse. The problem of obtaining satisfactory parental co-operation in a badly housed working-class population was apparent, and ways of reducing the demands that treatment imposes on parents are indicated. The results are discussed with regard to theories of conditioning treatment of nocturnal enuresis, and possible improvements in this technique of treatment.     

Control of variation by reward probability

Two bar-press experiments with rats tested the rule that reducing expectation of reward increases the variation from which reward selects. Experiment 1 used a discrete-trial random-interval schedule, with trials signaled by light or sound. One signal always ended with reward; the other signal ended with reward less often. The 2 signals were randomly mixed. Bar-press duration (how long the bar was held down) varied more during the signal with the lower probability of reward. Experiment 2 closely resembled Experiment 1 but used a random-ratio schedule rather than a random-interval schedule. Again, bar-press duration varied more during the signal with the lower probability of reward. The results support the rule--the first well-controlled comparisons to do so.     

A search for perceptual differentiation produced by nondifferential reinforcement

In each of four experiments (three using pigeons as the subjects and one using rats) animals in an experimental condition were given exposure to a pair of stimuli that subsequently they were required to discriminate between. During the exposure phase response to either stimulus was consistently followed by reward. These subjects were uniformly found to be retarded in learning the discrimination when compared with control subjects that received equivalent pre-training in the absence of the critical stimuli. These results provide no support for the suggestion that stimuli may become “perceptually differentiated” during the pre-training phase but are consistent with the proposal that novel stimuli possess a high level of associability that declines as a result of experience.     

The operation of an acquired drive in satiated rats

A series of four short experiments indicates that the behaviour of satiated rats in a runway, at the end of which they have previously been rewarded, differs significantly from the behaviour of satiated rats without previous reward in the experimental situation. The former group reach the end box more quickly after having been put in the starting box of the runway and if provided with food in the end box proceed to eat it, although they have just refused similar pieces of food in their home cages. This is shown not to be due to defective satiation or the operation of fear in the control group. When runs and feeding in the end box are separated during the training period, the previously rewarded group still shows a more vigorous response on satiated trials, indicating that it is a reward expectancy about the goal box rather than a running habit which has become “functionally autonomous”, acting as a situation-specific drive.     

Behavioural inhibition and the age at social isolation in rats

Rats that were socially isolated or group-housed at weaning were re-housed at 51 days of age in either the same housing condition or the opposite housing condition until 91 days and throughout subsequent behavioural testing. Increased resistance to extinction of running in an alley and poorer Hebb-Williams maze learning were found only after social isolation between 23 and 51 days of age. In contrast, slower running to reach the water reward during alley training and lower rearing activity at the beginning of open field testing were evident only in rats that were isolated at the time of testing. As in previous studies, open field hyperactivity was found in rats isolated at weaning but slower emergence latency was found in animals isolated at the time of testing. These findings confirm that the many behavioural effects of social isolation at different ages do not reflect a unitary aetiology. The behaviour of rats isolated at weaning is suggestive of a “disinhibition syndrome” but the precise nature of this syndrome is as yet unclear.     

Reward, novelty and spontaneous alternation

It has been suggested that rats' propensity for “win-shift” behaviour in spatial memory and spontaneous alternation tests reflects a species-specific foraging strategy which leads them to avoid places where they have recently found food. An alternative explanation is that they avoid places which are familiar. In three experiments using a T-maze, we evaluated these accounts by comparing the probability of avoiding or re-entering a recently visited arm, as a function of whether food had or had not been found on the previous visit. Each rat received a series of 16 exposure-test trial pairs over 8 days. Neither alternation nor repetition of the previous choice was differentially reinforced. Experiments I and II forced rats to enter a specific arm before a subsequent choice, and differed in the overall probability of reward; in Experiment III all choices were free. In all three experiments the probability of alternating was greater after nonreward than after reward. This effect occurred more reliably on later tests within a day, little difference appearing on earlier tests. It was concluded that there was no evidence for a spontaneous “win-shift” tendency as such, and that these and other results can be adequately accounted for by a combination of exploratory tendencies (spontaneous alternation) and the conventional effects of reward.     

Stimulus control of responding in the early trials of differential conditioning

In Experiment 1 two groups of rats were given 12 differential conditioning trials, seven to the rewarded alley (S+) and five to the nonrewarded alley (S?), prior to being extinguished in both alleys. Group S?S+ received S+ trials, following S? trials in acquisition, while Group S+S? did not receive S+ trials following S? trials in acquisition. In extinction S+ and S? trials were presented according to a quasi-random sequence for both groups. Running on the last 3 trials of acquisition was found to be faster following S+ than following S? trials. Group S?S+ showed greater resistance to extinction and less discriminative responding in extinction than Group S+S?. These results suggest that responding in differential conditioning is controlled not merely by S+ and S? but by the memories of reward (S^R) and of nonreward (S^N) as well. When the joint effects of both classes of cues were considered, e.g., S^R+S+, responding in the early trials of differential conditioning was shown to be highly orderly. Experiment 2 was highly similar to Experiment 1 except that Groups S?S+ and S+S? were equated along dimensions not equated in Experiment 1. The results obtained in Experiment 2 were highly similar to those obtained in Experiment 1.     

Age differences in the neural response to negative feedback

Affective processing is one domain that remains relatively intact in healthy aging. Investigations into the neural responses associated with reward anticipation have revealed that older and younger adults recruit the same midbrain reward regions, but other evidence suggests this recruitment may differ depending on the valence (gain, loss) of the incentive cue. The goal of the current study was to examine functional covariance during gain and loss feedback in younger and healthy older adults. A group of 15 older adults (mean age = 68.5) and 16 younger adults (mean age = 25.4) completed a revised Monetary Incentive Delay task (rMID; Knutson, Westdorp, Kaiser, &; Hommer, 2000) while in the fMRI scanner. The rMID is a reaction time task where successful performance, either gaining a reward or avoiding a loss, is defined by hitting a button during the brief presentation of a visual target. Participants receive gain and loss anticipation cues before each trial and feedback after each trial with four possible outcomes: +$5.00, +0.00, -$5.00, and -Affective processing is one domain that remains relatively intact in healthy aging. Investigations into the neural responses associated with reward anticipation have revealed that older and younger adults recruit the same midbrain reward regions, but other evidence suggests this recruitment may differ depending on the valence (gain, loss) of the incentive cue. The goal of the current study was to examine functional covariance during gain and loss feedback in younger and healthy older adults. A group of 15 older adults (mean age = 68.5) and 16 younger adults (mean age = 25.4) completed a revised Monetary Incentive Delay task (rMID; Knutson, Westdorp, Kaiser, & Hommer, 2000) while in the fMRI scanner. The rMID is a reaction time task where successful performance, either gaining a reward or avoiding a loss, is defined by hitting a button during the brief presentation of a visual target. Participants receive gain and loss anticipation cues before each trial and feedback after each trial with four possible outcomes: +$5.00, +0.00, -$5.00, and -$0.00. Using seed-voxel partial least squares analyses, with seed voxels in the caudate and ventromedial prefrontal cortex, whole-brain functional covariance revealed that younger and older adults engage the same network of regions to support general feedback processing. However, older adults engaged two additional networks to support processing of negative feedback, gain_miss (+0), loss_miss (-$5), and loss_hit (?0), specifically. These findings are in line with theories of a positivity effect in aging and may have implications for reward-stimulus learning and decision making following performance-contingent negative feedback.     

Orbital prefrontal cortex and guidance of instrumental behavior of rats by visuospatial stimuli predicting reward magnitude

The orbital prefrontal cortex (OPFC) is part of a circuitry mediating the perception of reward and the initiation of adaptive behavioral responses. We investigated whether the OPFC is involved in guidance of the speed of instrumental behavior by visuospatial stimuli predictive of different reward magnitudes. Unoperated rats, sham-lesioned rats, and rats with bilateral lesions of the OPFC by N-methyl-D-aspartate (NMDA) were trained in a visuospatial discrimination task. The task required a lever press on the illuminated lever of two available to obtain a food reward. Different reward magnitudes were permanently assigned to lever presses to respective sides of the operant chamber; that is, responses to one lever (e.g., the left one) were always rewarded with one pellet and responses to the other lever with five pellets. On each trial, the position of the illuminated lever was pseudorandomly determined in advance. Results revealed that OPFC lesions did not impair acquisition of the task, as the speed of conditioned responses was significantly shorter with expectancy of a high reward magnitude. In addition, during reversal, shift and reshift of lever position–reward magnitude contingencies and under extinction conditions, performance of the OPFC-lesioned and control groups did not differ. It is concluded that the OPFC in rats might not be critical for adapting behavioral responses to changes of stimulus–reward magnitude contingencies signaled by visuospatial cues.     

“We will work for you” – Social influence may suppress individual food preferences in a communicative situation in dogs

The level of motivation (i.e. incentive power) is thought to be one of the most important factors affecting performance and learning in various tasks. We investigated whether reward quality has an effect on the performance of family dogs in a two-way object choice test in which they can find the hidden food by relying on distal momentary human pointing cues. In three experiments we varied (1) the type of food reward according to the subjects’ own preference; (2) the quality of the reward offered at the same time in the indicated and not-indicated locations; and (3) the order of the high or low quality rewards in consecutive sessions. In Experiment 1, we first tested whether dogs prefer one kind of reward over another. Then one group was tested with the ‘preferred’ food as reward in the indicated bowl, while dogs in the other group received the ‘non-preferred’ food as reward. We did not find any difference between the performance and choice latencies of the two groups. In Experiment 2 for the first group, the indicated bowl contained a piece of carrot and the not-indicated bowl was empty. In the second group the indicated bowl contained carrot, but the not-indicated bowl contained sausage. According to a preliminary preference test, most dogs prefer sausage over carrot invariably. After 20 trials, the two groups performed surprisingly similarly. There was no difference found between groups in the number of correct choices, incorrect choices and non-choices. However, the comparison between the first and last five trials revealed that subjects who found sausage when they chose the not-indicated bowl (did not follow the pointing) chose the non-indicated bowl significantly more often toward the end of their test session. In Experiment 3, each dog received two sessions with 12 pointing trials in each. For the first session, one group was rewarded with sausage and the other with carrot upon choosing the indicated bowl. In the second session, the indicated bowl contained dry dog food for both groups. We found that correct choices and response latencies did not change over two sessions in the ‘sausage’ group. In the ‘carrot’ group, the dogs chose faster in the second session, but their performance did not improve; in fact, they chose the not-indicated bowl more often than the indicated bowl. As a conclusion, we can say that reward quality had some effect on dogs’ choice behavior in these experiments. The drop in their performance was not drastic, taking into account the general refusal to eat one of the ‘rewards’ (carrot) during the preference tests and also during the test trials. It seems that incentive contrast may play a relatively minor role in dog-human social interactions. Appropriate reward quality can be very important in asocial problem solving tasks, but, when interacting with humans, following human signals may override the effect of changed incentive power.     

Resistance to continuous delay of reinforcement or extinction following partial delay or partial reinforcement in acquisition: A direct comparison

Rats received either partial reward (PR) or partial delay (PD) in acquisition with one, two, or three delay or nonreward trials followed by an immediately rewarded trial or one delay or nonreward trial followed by an immediately rewarded trial. These four groups were then split in half and given either continuous delay or continuous nonreward (extinction) in a “response persistence” phase. In addition, two continuously reinforced groups, one experiencing continuous delay, and the other experiencing extinction were included. The results showed that response persistence was greater when PD groups were given continuous delay rather than extinction, but the opposite was true for PR groups. The “length” (1, 2, or 3 versus 1 nonreward or delay trial) also transferred to the response persistence phase with the length 1, 2, and 3 conditions being more persistent than the 1 length conditions. The results were discussed with respect to theoretical notions of response persistence.     

Preference for the outcome that follows a relatively aversive event: Contrast or delay reduction?

Pigeons prefer a positive discriminative (S+) stimulus that follows a less preferred event (a large number of required responses, a longer delay, or the absence of food) over a different S+ with a similar history of reinforcement that follows a more preferred event (a single required response, no delay, or food). We proposed that this phenomenon results from contrast (referred to as within-trial contrast) between the less preferred initial event and the signal for reinforcement. Delay reduction theory (Fantino, 1969) can account for these results by proposing that the less preferred initial event lengthens the duration of the trial, thereby allowing the S+ stimulus to occur later in the trial and thus become a better predictor of reinforcement. In the present experiments, we further explored this effect. In Experiment 1, we controlled for trial duration by using a fixed ratio response (30 pecks) as one initial event and the absence of pecking for the same duration as the other initial event (0 pecks). The pigeons showed a reliable preference for the positive stimulus that followed the least preferred initial event. In Experiment 2, we controlled for trial duration by using 30 pecks as one initial event and 1 peck followed by a delay that matched the duration of the preceding 30-peck trial. (Group Time Same). For Group Time Different, there was no delay following the 1-peck initial event. For Group Time Same, preference for the initial event negatively predicted the pigeons’ preference for the S+ stimulus that followed, supporting the contrast account. A somewhat greater preference for the discriminative stimulus that followed the least preferred initial event was found for Group Time Different suggesting that in addition to contrast, delay reduction also may play a small role. However, the greater initial-event preference found for Group Time Different suggests that contrast can account for the group difference as well.     

Reward anticipation enhances brain activation during response inhibition

The chance to achieve a reward starts up the required neurobehavioral mechanisms to adapt our thoughts and actions in order to accomplish our objective. However, reward does not equally reinforce everybody but depends on interindividual motivational dispositions. Thus, immediate reward contingencies can modulate the cognitive process required for goal achievement, while individual differences in personality can affect this modulation. We aimed to test the interaction between inhibition-related brain response and motivational processing in a stop signal task by reward anticipation and whether individual differences in sensitivity to reward (SR) modulate such interaction. We analyzed the cognitive–motivational interaction between the brain pattern activation of the regions involved in correct and incorrect response inhibition and the association between such brain activations and SR scores. We also analyzed the behavioral effects of reward on both reaction times for the “go” trials before and after correct and incorrect inhibition in order to test error prediction performance and postinhibition adjustment. Our results show enhanced activation during response inhibition under reward contingencies in frontal, parietal, and subcortical areas. Moreover, activation of the right insula and the left putamen positively correlates with the SR scores. Finally, the possibility of reward outcome affects not only response inhibition performance (e.g., reducing stop signal reaction time), but also error prediction performance and postinhibition adjustment. Therefore, reward contingencies improve behavioral performance and enhance brain activation during response inhibition, and SR is related to brain activation. Our results suggest the conditions and factors that subserve cognitive control strategies in cognitive motivational interactions during response inhibition.     

Successive positive contrast in one-way avoidance learning

The main finding of these experiments was a positive contrast effect in one-way avoidance learning. Experiment 1 showed that increasing safety time during one-way avoidance training led to improved performance, surpassing that of a control group that had received the high reward (safe time) from the beginning of training. Experiment 2 showed that a similar positive contrast effect occurred when the time spent in the danger compartment before the onset of the warning signal was shortened. These results suggest that time spent in a safe context acts as a reinforcer of the avoidance response; however, its incentive value depends not only on its duration, but also on the length of the time spent in the danger compartment before the onset of the signal. Overall, results also suggest that the avoidance response is a mixture of flight (motivated by fear) and approach (to a safe place) behaviour. The specific weight of the flight or approach component may be a function of the time and the amount of activation of each emotional state (fear or relief) due to opponent homeostatic compensatory processes that occur in the danger and safe compartments during one-way avoidance learning.