Transfer of approach responding between punishment, frustrative nonreward, and the combination of punishment and nonreward

Rats trained to make an approach response with either partial reward, intermittent punishment, or a combination of partial reward and intermittent punishment, were tested for persistence to extinction, punishment with reward, or punishment during extinction. Partial reward, alone or with punishment, produced greates resistance to extinction, while intermittent punishment, alone or with partial reward, produced greatest persistence to punishment with reward. Transfer of persistence from partial reward to punishment with reward and intermittent punishment to extinction was also demonstrated. However, partial reward alone did not increase persistence to punishment during extinction, whereas intermittent punishment and partial reward combined with intermittent punishment did increase such persistence. These results were interpreted in Amsel's (1958, 1962) conditioning-model theory by extending the hypothesized similarity of frustrative nonreward and punishment.     

The order of continuous,partial and nonreward trials and resistance to extinction

In the first experiment rats were given either 16 or 48 nonrewarded or continuously rewarded trials prior to 24 continuously or partially rewarded trials, followed by extinction. Increased resistance to extinction was found for increasing numbers of nonrewarded trials when they were followed by partial reward, but not when followed by continuous reward. Similarly, more continuously rewarded trials followed by partial reward tended to increase resistance to extinction. Because of the theoretical importance of the effect of continuous reward followed by partial, a second experiment was performed where the range of the number of continuously rewarded trials was extended to 0, 48, and 96. Contrary to many theoretical predictions, resistance to extinction increased as a function of increasing amounts of continuous reward.     

Odor production and utilization under conditions of nonreward and small reward

A series of five experiments was conducted to examine odor production and utilization under conditions of contrasting large and small reward. Using daily, eight-trial double-alternation sequences of reward (L)-nonreward (N) or small reward (S), i.e., LLNNLLNN and LLSSLLSS, appropriate patterning was shown only when an empty goalbox was initially encountered on N or S trials (Experiments 1–5). Manipulations designed to increase frustration, for example, preceding double-alternation training with a block of large-reward trials in Experiment 2, or increasing the L vs S ratio in Experiment 3, failed to yield patterning when small reward was presented at the start of the S-trial confinement period. Patterning was shown when small reward was presented at the end of the S-trial confinement (Experiments 4 and 5). Immediate vs delayed (30 sec) S or N trial goalbox removal failed to result in differences in patterning (Experiments 1 and 2). To distinguish odor production from odor use, subjects trained on an LLSSLLSS pattern preceded subjects trained on an LLNNLLNN pattern (Experiment 5). Because patterning failed to develop in the LLSSLLSS subjects and the first subject in the LLNNLLNN group, it was concluded that a discriminably different cue was not produced on S trials under the LLSSLLSS sequence when small reward was delivered at the beginning of the goalbox confinement period. Shifting small-reward delivery to the end of goalbox confinement resulted in the immediate display of patterning by the first LLNNLLNN subject, and the subsequent development of patterning by the LLSSLLSS subjects.     

Effects of reward and nonreward on frustration and attention in attention deficit disorder

The effects of reward schedule (100%, 50%, and 30%) and termination of rewards (extinction) on 30 attention deficit disorder with hyperactivity (ADD-H) and 30 normal children were studied using measures of frustration (speed/strength of lever pulling) and attention (reaction time to a light signal). ADD-Hs pulled harder on the lever than controls during extinction and on the lowest (30%) partial schedule, providing empirical evidence that they respond with greater frustration than normals when expected rewards fail to appear. The groups did not differ on the attentional measure on 100% reward. However, the partial schedules appeared to have an alerting or motivating effect on the controls, so that they responded more quickly and consistently than ADD-Hs on the partial schedules. Findings are discussed with reference to opposing theories regarding the nature of the abnormal response of ADD-Hs to reward.This research was supported by grant MA 11252 from the Medical Research Council of Canada.     

The effects of post-acquisition-trial sodium amylobarbitone on subsequent extinction behaviour

In a one-trial-per-day runway task, previously trained rats were injected via temporary intravenous cannulae with Sodium Amylobarbitone (SA) 15 mg/kg, or saline, immediately following each of six acquisition trials, according to one of four regimes. The animals were then extinguished with no further drug or saline treatments. Those that had been consistently reinforced (CR) and consistently treated with saline extinguished faster than those that had been partially reinforced (PR) and consistently treated with saline: the brief PR schedule thus generated a typical PR extinction effect. PR animals that had received SA treatments following nonrewarded trials and saline treatments following rewarded trials extinguished faster than those that had received the opposite treatments. Post-nonreward SA treatment during acquisition thus led to a pseudo CR extinction effect, while post-reward SA treatment in acquisition led to an undistorted PR extinction effect. The results are interpreted as showing that there is a critical period occurring between some tens of seconds and a very few minutes after frustrative nonreward which is vulnerable to SA in a non-state-dependent manner, and on which frustrative nonreward's control of subsequent behaviour depends.     

Transfer of approach responding between punishment and frustrative nonreward sustained through continuous reinforcement

A block of continuously reinforced nonpunished trials was interpolated between acquisition of a runway response with either partial reinforcement or intermittent punishment and subsequent tests for resistance of that response to the suppressive effects of either extinction or continuous punishment. As previous investigations have shown, both the partial reinforcement effect (PRE) and the intermittent punishment effect (IPE) were sustained through the block of continuously reinforced nonpunished trials. Furthermore, the increased resistance to extinction following intermittent punishment and the increased resistance to punishment following partial reinforcement were also sustained through the interpolated continuous reinforcement. These results support a hypothesized similarity of punishment and frustrative nonreward and were interpreted in an extension of Amsel's conditioning model theory of the role of nonreinforcement in the PRE.     

Cues produced by reward and nonreward and temporal cues influence responding in the intertrial interval and to the conditioned stimulus

Rats in a Pavlovian situation were trained under three different reward schedules, at either a 30 s or a 90 s intertrial interval (ITI): Consistent reward (C), 50% irregular reward (I), and single alternation of reward and nonrewarded trials (SA). Activity was recorded to the conditioned stimulus (CS) and in all 10 s bins in each ITI except the first 10 s bin. Responding increased progressively over the bins of the 90 s ITI indicating a role for temporal cues in the ITI. Single alternation patterning (SAP), more vigorous responding on rewarded than on nonrewarded trials, occurred in the ITI as well as to the CS and differed little at the 30 s and 90 s ITI. The occurrence of SAP to the CS and, as shown here for the first time, in the ITI along with the findings for the C and I schedules, indicate three matters of importance. First, along with other cues such as temporal cues, behavior is influenced by cues associated with reward and nonreward both in the ITI and when the CS is presented. Second, as the percentage of reward associated with these cues increases so too does vigor of responding. Finally, if the cues produced by reward events are memories the retrieval cues for such memories are present in the ITI as well as when the CS is presented. Considerations are advanced indicating that reward produced cues may influence responding in the Pavlovian situation not only under regular schedules such as the SA schedule, but under all schedules. Indeed, such cues may more validly signal reward and nonreward than the CS itself.     

The other side of the coin: Cognitive properties of nonreward

In this comment, I examine Amsel’s theory in the light of rats’ reaction not to frustrative non-reward per se, but to the more complicated case in which frustrative nonreward is part of a rule-based sequence of reward quantities. The discussion goes beyond Amsel’s emphasis on dispositional memory to consider cognitive and representational memory—the signaling properties of reinforcement and nonreinforcement. The strengths and weaknesses of Amsel’s theory are discussed with the fresh perspective that this viewpoint provides, and some issues emerge that may be fruitful for further study.