Repeated acquisition in the analysis of rule-governed behavior

Five children, ranging in age from 3½ years to 5½ years, were taught various four-response chains using conditioned reinforcement. Experiment 1 investigated the effects of presenting “instruction” stimuli—a sequence of lights over the correct response buttons—to assess their role in facilitating the acquisition of a chain of responses. Without the “instruction” stimuli, children made many errors before responses were brought under the control of the programmed contingencies. When confronted with the same contingencies later in the day, these subjects made fewer errors. In contrast, in the presence of the “instruction” stimuli, subjects made virtually no errors. However, when the “instruction” stimuli were discontinued in the subsequent session, all 5 subjects made errors. In Experiment 2, the subjects were taught to verbalize the contingencies during the phase without the “instruction” stimuli. This resulted in errorless performance during the subsequent exposure to the same procedure, but errors nevertheless occurred again during reexposure to the procedure with the “instruction” stimuli discontinued.     

Selective attention: the effects of combining stimuli which control incompatible behavior

Four rhesus monkeys learned both a color and tilt discrimination. The stimuli were combined to produce incompatible behavior. The behavior controlled by one set of stimuli was reinforced until “errors” virtually disappeared. The stimuli were tested separately again. Sixteen replications of the entire procedure indicated that the stimuli producing “errors” were ignored.     

Stimulus control in the use of landmarks by pigeons in a touch-screen task

Pigeons were tested in a search task on the surface of a monitor on which their responses were registered by a touch-sensitive device. A graphic landmark array was presented consisting of a square outline (the frame) and a colored “landmark.” The unmarked goal, pecks at which produced reward, was located near the center of one edge of the frame, and the landmark was near it. The entire array was displaced without rotation on the monitor from trial to trial. On occasional no-reward tests, the following manipulations were made to the landmark array: (a) either the frame or the landmark was removed; (2) either one edge of the frame or the landmark was shifted; and (3) two landmarks were presented with or without the frame present. On these two-landmark tests, the frame, when present, defined which was the “correct” landmark. When the frame was absent, the “correct” landmark was arbitrarily determined. Results showed that pecks of 2 pigeons were controlled almost solely by the landmark, pecks of 3 were controlled primarily by the landmark but the frame could distinguish the correct landmark, and 1 bird's behavior was controlled primarily by the frame. Stimulus control in this search task is thus selective and differs across individuals. Comparisons to other search tasks and to other stimulus control experiments are made.     

Orbital prefrontal cortex and guidance of instrumental behavior of rats by visuospatial stimuli predicting reward magnitude

The orbital prefrontal cortex (OPFC) is part of a circuitry mediating the perception of reward and the initiation of adaptive behavioral responses. We investigated whether the OPFC is involved in guidance of the speed of instrumental behavior by visuospatial stimuli predictive of different reward magnitudes. Unoperated rats, sham-lesioned rats, and rats with bilateral lesions of the OPFC by N-methyl-D-aspartate (NMDA) were trained in a visuospatial discrimination task. The task required a lever press on the illuminated lever of two available to obtain a food reward. Different reward magnitudes were permanently assigned to lever presses to respective sides of the operant chamber; that is, responses to one lever (e.g., the left one) were always rewarded with one pellet and responses to the other lever with five pellets. On each trial, the position of the illuminated lever was pseudorandomly determined in advance. Results revealed that OPFC lesions did not impair acquisition of the task, as the speed of conditioned responses was significantly shorter with expectancy of a high reward magnitude. In addition, during reversal, shift and reshift of lever position–reward magnitude contingencies and under extinction conditions, performance of the OPFC-lesioned and control groups did not differ. It is concluded that the OPFC in rats might not be critical for adapting behavioral responses to changes of stimulus–reward magnitude contingencies signaled by visuospatial cues.     

General attentiveness effects of discriminative training

Using a design that permitted the simultaneous assessment of intra-, inter-, and extradimensional effects of discriminative training, the generality of discriminative effects that have been said to reflect increases in “general attentiveness” was assessed. Pigeons received either discriminative training with two stimuli correlated with reinforcement and one stimulus correlated with nonreinforcement, or nondifferential reinforcement (control) training. One positive stimulus was part of an intradimensional task and the other was not. After training, generalization tests were conducted to assess stimulus control along several dimensions. Discriminative training resulted in increased control along dimensions of the positive stimulus involved in the intradimensional task, but not along any dimensions of the other positive stimulus. The results suggested that discriminative training leads to increases in attention that are neither so general as suggested by the “general attentiveness” view nor so specific as to be revealed solely by intradimensional effects.     

Working Memory for Temporal and Nontemporal Events in Monkeys

This is the first report that introduces appropriate behavioral tasks for monkeys for investigations of working memory for temporal and nontemporal events. Using several behavioral tests, the study also shows how temporal information is coded during retention intervals in the tasks. Each of three monkeys was trained with two working memory tasks: delayed matching-to-sample of stimulus duration (DMS-D) and delayed matching-to-sample of stimulus color (DMS-C). The two tasks employed an identical apparatus and responses and differed only in the temporal and nontemporal attribute of the stimuli to be retained for correct performance. When a retention interval between the sample and comparison stimuli was prolonged, the monkeys made more incorrect responses to short samples in the DMS-C task, suggesting “trace decay” of memory for short stimuli. However, the same monkeys showed no such increase in incorrect responses to short samples in the DMS-D task, suggesting active coding of temporal information, that is, the length of stimulus duration, during the retention interval. When variable lengths of samples were presented with a fixed retention interval, the monkeys made more incorrect responses when length differences between short and long samples were small in the DMS-D task, but not in the DMS-C task. This suggests that the codes of working memory retained in the DMS-D task were not absolute (analogical) but rather were relative (categorical) and related to differences in the duration of the samples.     

Recognition by the pigeon of stimuli varying in two dimensions

Pigeons served in four experiments, each of which involved about 44,000 discrete 1.2-sec trials under steady-state conditions. The first experiment scaled a short segment of the visual wavelength continuum; this dimension was then combined in a conditional discrimination with each of three others; time after reinforcement, tone frequency, and line tilt. In the two-stimulus experiments, the birds' responses were reinforced in the presence of only one stimulus combination: “582 nm” together with “2 min after reinforcement”, “3990 Hz”, or “vertical line”. Many other stimulus combinations also appeared equally often and went without reinforcement. The wavelength stimuli conformed to an equal-interval scale, and per cent response was generally linear with wavelength, when scaled on cumulative normal coordinates. The components of the compound stimulus were found to interact in a multiplicative fashion; when one component differed greatly from its reinforcement value, changes in the other component had relatively little effect. For the “time”-“wavelength” compound, this interaction appeared to be modified by the effects of set or attention. Certain response latency data are reported, and other combination rules are discussed.     

The effect of overtraining on behavioral contrast and the peak-shift

Following initial discrimination training between two wavelength stimuli and a subsequent generalization test to the wavelength dimension, Group 1 was “overtrained” for 105 days on the original discrimination. Group 2 was “overtrained” with the original positive stimulus and a new negative stimulus, a white line. Group 3 was “overtrained” with the original negative stimulus and a new positive stimulus, the white line. Each 15 days of extended training were followed by a wavelength generalization test similar to the first test. The results suggest that there is no consistent relationship between the response rate in positive stimulus immediately before the generalization test and whether or not a peak shift occurs during the test.     

Reward prediction error signals by reticular formation neurons

As a key part of the brain’s reward system, midbrain dopamine neurons are thought to generate signals that reflect errors in the prediction of reward. However, recent evidence suggests that “upstream” brain areas may make important contributions to the generation of prediction error signals. To address this issue, we recorded neural activity in midbrain reticular formation (MRNm) while rats performed a spatial working memory task. A large proportion of these neurons displayed positive and negative reward prediction error-related activity that was strikingly similar to that observed in dopamine neurons. Therefore, MRNm may be a source of reward prediction error signals.The capacity of an organism to respond appropriately to environmental stimuli depends on the ability to detect changes in the outcome of its behavior. The mesocorticolimbic dopamine system is thought to be central to this function (Wise 2004; Fields et al. 2007). Dopamine neurons in the ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) increase activity relative to the presentation of cues that predict rewards and rewards of greater value than expected, and decrease activity relative to rewards of less value than predicted (Nakahara et al. 2004; Bayer and Glimcher 2005; Pan et al. 2005; Tobler et al. 2005). This activity is thought to be involved in a computation about errors in the prediction of reward (Schultz and Dickinson 2000) that can be used to correct behavior. A central issue relevant to the behavioral and computational interpretation of dopamine signals is whether prediction error signals are generated by dopamine neurons, per se, or by cells in “upstream” brain areas.Recent data suggest that brain areas afferent to dopamine neurons generate, or participate in, reward prediction error computations. Lateral habenula, which has been shown to inhibit the activity of VTA and SNc dopamine neurons (Herkenham and Nauta 1979; Christoph et al. 1986), has recently been identified as a potential source of reward prediction error signals (Matsumoto and Hikosaka 2007). A similar finding has been demonstrated in the pedunculopontine tegmental nucleus (PPTg), which is an important regulator of dopamine neuron activity (Floresco et al. 2003). PPTg neural responses varied according to whether or not the animal received expected rewards (Kobayashi and Okada 2007).As part of a larger study investigating the role of VTA in context-dependent spatial working memory (C.B. Puryear, M.J. Kim, and S.J.Y. Mizumori, in prep.), we recorded the activity of neurons in the magnocellular region of the midbrain reticular formation (MRNm according to the method of Swanson [2003]) at the level of the diencephalon. The reticular formation is thought to be important for modulating arousal and vigilance levels necessary for attending to and acting upon salient stimuli (Pragay et al. 1978; Mesulam 1981), and it has recently been shown that this portion of the reticular formation provides glutamatergic input to VTA (Geisler et al. 2007). Thus, MRNm is in a prime position to modulate the activity of VTA dopamine neurons when the outcome of behavior does not meet expectations and therefore may be a source of reward prediction error signals. Accordingly, we investigated whether MRNm neurons exhibited reward related activity, and whether this activity was related to the ability to predict acquisition of reward.Four male Long-Evans rats (4–6 mo old from Simonson Laboratory, Gilroy, CA) were housed individually in Plexiglas cages in a temperature- and humidity-controlled environment (12:12 h light:dark). Rats were provided with food and water ad libitum for 5 d prior to being handled daily and reduced to 85% of ad libitum feeding weights. Animal care and use was conducted according to University of Washington’s Institutional Animal Care and Use Committee guidelines.Rats were habituated to the testing environment and trained to perform a differential reward, win-shift spatial working memory task using radial maze procedures reported previously (Pratt and Mizumori 2001; C.B. Puryear, M.J. Kim, and S.J.Y. Mizumori, in prep.). Briefly, prior to the start of each trial, the end of each of the eight maze arms was baited with either a large (five drops) or small (one drop) amount of chocolate milk on alternating arms. Maze arms containing large or small amounts of reward were counterbalanced across rats and held constant throughout training. Trials started with a sample phase by presenting four maze arms (two large and two small reward arms; individually and randomly selected) to the rat. Immediately after presentation of the fourth arm, a test phase began by making all maze arms accessible so the rat could collect the remaining rewards. The trial ended once all arms were visited. The rat was then confined to the center of the maze for a 2-min intertrial interval. Arm re-entries were counted as errors. Once the rat was able to perform at 15 trials in less than 1 h for seven consecutive days, recording electrodes were surgically implanted.Details concerning the construction of recording electrodes and microdrives and surgical procedures can be found in previous works (McNaughton et al. 1983; Puryear et al. 2006; C.B. Puryear, M.J. Kim, and S.J.Y. Mizumori, in prep.). Briefly, rats were chronically implanted with either eight stereotrodes (four/hemisphere) or four tetrodes (two/hemisphere) made from 25-μm lacquer-coated tungsten wire (California Fine Wire), centered on the following coordinates relative to bregma: −5.25 mm posterior, 0.7 mm lateral, 6 mm ventral. One week of free feeding was allowed for rats to recover from surgery before recording experiments began.Recordings were performed as described previously (Puryear et al. 2006; C.B. Puryear, M.J. Kim, and S.J.Y. Mizumori, in prep.). If no clear spontaneous neural activity was encountered, electrodes were lowered in ∼25-μm increments (up to 175 μm/d) until unambiguous, isolatable units were observed. Single units were isolated from multiunit records using standard cluster-cutting software (MClust; A.D. Redish, University of Wisconsin). A template-matching algorithm was also used to facilitate separation of unique spike waveforms. We only included cells that had a high signal-to-noise ratio (>3:1), exhibited stable clusters throughout the recording session, and had clear refractory periods in the interspike interval histograms following cluster cutting.The final position of each stereotrode was marked by passing a 25-μA current through each recording wire for 25 sec while rats were under 5% isofluorane anesthesia. Rats were then given an overdose of sodium pentobarbital and transcardially perfused (0.9% buffered saline, followed by 10% formalin). Electrodes were retracted, and the brain was removed and allowed to sink in 30% sucrose-formalin. Coronal sections (40 μm) were sliced with a cryostat and stained with cresyl violet. Recording locations were verified by comparing depth measurements and reconstructions of the electrode tracks. Only cells determined to be located in MRNm (Swanson 2003) were considered for analysis.Rats were well trained on the spatial working memory task, committing 0.86 ± 0.2 (mean ± SEM) errors per trial during the first five trials (baseline trials). Importantly, rats demonstrated the ability to discriminate large and small reward locations. There was a significant negative correlation between the first four test phase arm choices (i.e., first, second, third, and fourth arm choice) and the probability that the arm chosen contained a large reward (Fig. 1B; Spearman’s ρ = −0.65, P < 0.001), indicating that rats reliably visited large reward arms before small reward arms during the test phase of each trial.Open in a separate windowFigure 1.Histology and basic firing properties of MRNm neurons. (A) Distribution of cells localized to MRNm. Each dot may represent the location of more than one neuron. Coronal slices adapted from Swanson (2003) (reprinted with permission from Academic Press ©2003). (B) Rats displayed preference for arms that contained large amounts of reward. Plotted is the average probability of choosing a large reward arm during the first four arm choices of the test phase of each trial. Error bars represent SEM. (C) Distribution of average firing rates and spike duration of MRNm neurons. Most cells fired less than 10 spikes/sec and exhibited waveform durations between 1.5 and 2.0 msec. (D) Examples of two MRNm neurons. Top row shows their average waveform on each wire of the tetrode. Scale bar = 1 msec. Middle and bottom rows depict their interspike interval and autocorrelation histograms, respectively.A total of 18 cells localized to MRNm were recorded while rats performed the task. Of these, one cell was omitted from analysis due to a very low average firing rate (∼0.2 spikes/sec), yielding 17 cells included in the following analyses. Figure 1A depicts the distribution of cells localized to MRNm. These cells exhibited a range of average firing rates, spike durations (defined as the time from the start to the end of the action potential) (Fig. 1C), and firing patterns (for representative interspike interval and autocorrelation histograms, see Fig. 1D).Reward-related neural activity was obtained by placing rewards in small metal cups mounted to the end of each maze arm and connected to the recording equipment (custom designed by Neuralynx, Inc.), which served as “lick detectors.” An event marker was automatically inserted into the data stream when the rat licked the cup, providing an instantaneous measurement of the time the rat first obtained reward.In order to determine whether MRNm neurons exhibited significant reward-related activity, peri-event time histograms (PETHs) were constructed (50 msec bins, ±2.5 sec around each reward event). A cell was considered to have a significant excitatory reward response if it passed the following two criteria: (1) The cell had a peak firing rate within ±150 msec of reward acquisition and (2) the peak rate was >150% of its average firing rate for the block of trials. These criteria were applied to PETHs collapsed across reward amounts and separately for large and small reward events. Overall, 47% (eight of 17) of MRNm neurons were found to exhibit significant excitatory responses upon acquisition of reward (Fig. 2D). Of these cells, most (88%, seven of eight) were found to fire relative to acquisition of only large rewards (e.g., Fig. 2A–C), while the remaining neuron fired relative to acquisition of both reward amounts. No cells were found to fire preferentially to acquisition of only small rewards. Aspects of animals’ movement (e.g., velocity) were not found to be a major contributor to the firing patterns of MRNm neurons during performance of the spatial working memory task (data not shown). Therefore, it appears that MRNm unit activity is predominantly biased to represent higher reward values.Open in a separate windowFigure 2.Reward-related activity of MRNm neurons. (A) Peri-event time histograms of one cell that exhibited a short-latency, excitatory response upon acquisition of rewards (t₍₀₎, bin width = 50 msec). Left histogram shows only a modest excitatory response when considering all rewards together. However, top and bottom right histograms show that the reward-related firing occurred upon acquisition of large and not small rewards. Gray-shaded areas indicate time periods analyzed for significant increases in firing rate. (B) Population summary of the proportion of MRNm neurons that demonstrated significant reward-related activity.In order to determine whether MRNm reward-related activity was associated with reward prediction, we tested unit responses to unexpected alterations of reward outcome or elimination of visuospatial information important for reward prediction. To do this, we allowed the rat to perform a second block of five trials with either the locations of large and small rewards switched (reward location switch condition), with two rewards (one large and one small, randomly selected) omitted from the study phase of each trial (reward omission condition), or with the maze room lights extinguished (darkness condition). Importantly, each of these manipulations created situations in which reward prediction errors likely occurred. Overall, these three testing conditions created the following situations, respectively: a mismatch between the locations of large and small rewards, a decreased probability of obtaining a reward, and a situation in which rats are not able to discriminate between arms associated with large and small amounts of reward (C.B. Puryear, M.J. Kim, and S.J.Y. Mizumori, in prep.). Therefore, positive prediction errors could occur when the animal received a large amount of reward on an arm previously associated with a small amount, when the rat retrieved rewards after visiting arms in which reward had been omitted, and when the rat obtained a large reward in darkness. Negative reward prediction errors could occur when the animal received a small amount reward on a maze arm previously associated with a large amount, when the rat visited an arm that did not contain a reward, and when the rat obtained a small reward in darkness.Eight cells with significant responses to large rewards were recorded during these tests (two during reward location switch, four during reward omission conditions, and two during darkness conditions). In order to determine whether the reward manipulations affected the reward-related activity of these cells, a reward activity value (RA) was first calculated, which was the average firing rate in the ±150 msec around the time of acquisition of rewards, expressed as a percentage in change relative to the cell’s average firing rate for each block of trials. These values were normalized to the maximum RA value observed, yielding a normalized RA value for the first and second block of trials (RA_n1 and RA_n2, respectively). These calculations were made for large and small rewards separately, and for non-rewarded arms in the reward omission condition.We then created scatterplots of RA_n’s for each block of trials. If reward-related activity was independent of the expectation of the reward received, RA_n1 and RA_n2 should be similar in each block of trials. As can been seen in Figure 3A, the reward-related activity was consistently higher when rats received more reward than expected in the second block of trials. Conversely, Figure 3B clearly shows that neural activity was consistently suppressed when rats received less reward than expected. These differences in reward-related firing were quantified by calculating the distance of each data point to the diagonal (i.e., the reward activity change index, or RACI): Directionality of the change in reward activity was taken into account in order to discern between increases and decreases in firing rate. A one-sample t-test (α = 0.05) indicated that average RACI values were significantly increased when rats received more reward than expected (t₍₇₎ = −2.73, P < 0.03) and were significantly decreased when rats received less reward than expected (t₍₇₎ = −4.88, P < 0.001). These results are consistent with positive and negative reward prediction error signals, respectively. An example of an MRNm neuron that exhibited both positive and negative prediction error-related activity in the reward location switch condition is depicted in Figure 3D.Open in a separate windowFigure 3.Reward prediction errors in MRNm neurons. (A) Plotted is each neuron’s normalized large reward activity (RA_N, defined in text) for each block of trials. The reward activity during block 2 (y-axis) represents activity at times when more reward than expected was obtained. Note that reward-related activity during these times is consistently more robust than during times in which the rat received the expected reward (block 1), indicating that positive reward prediction errors occurred. (B) Plotted are RA_N values for rewarded and devalued arms in block 2 (x- and y-axes, respectively). Devalued arms include arms associated with a large amount of reward but baited with a small amount of reward, arms in which reward was omitted, and arms containing small rewards visited in darkness. Note that reward-related activity on devalued arms is consistently suppressed, indicating that negative reward prediction errors occurred. (Symbols in A,B: ● indicates cell recorded in darkness condition; o, cell recorded in reward omission condition; and x, cell recorded in reward location switch condition.) (C) Average changes in reward activity (RACI, defined in text) for times in which the rat obtained more and less reward than expected. Asterisks indicate significant differences (P < 0.05). Error bars indicate SEM. (D) An example of one neuron that did not respond to acquisition of rewards during the first block of trials. When the locations of large and small rewards were switched, however, the cell developed an excitatory response to acquisition of large rewards on arms previously associated with small amounts of reward (positive reward prediction error). Furthermore, the firing of the cell was inhibited upon acquisition of small rewards on arms previously associated with large amounts of reward (negative reward prediction error).We demonstrate here that a large proportion of MRNm neurons may be involved in computations about reward acquisition. Similar to dopamine neurons (Tobler et al. 2005), the majority of reward-related MRNm neurons preferentially fired relative to acquisition of large amounts of reward. To our knowledge, this is the first demonstration of discriminative reward responses of reticular formation neurons and highlights a novel role for the reticular formation in reward value representations. Furthermore, these data suggest that MRNm neurons represent similar reward prediction error signals as dopamine neurons (Nakahara et al. 2004; Bayer and Glimcher 2005; Pan et al. 2005; Tobler et al. 2005). It is important to note that in this initial sample, there was remarkable overall consistency and reliability of the positive and negative reward prediction error signals by MRNm neurons. This is similar to the homogeneity of dopamine neuron responses, suggesting that reward prediction may be a major function of the overall population of MRNm reward-related neurons. Nevertheless, further parametric studies are necessary to determine whether MRNm neural activity conforms to the same basic firing profiles that have been well-described for dopamine neurons (i.e., predictive cues and reward probabilities).The reticular formation has traditionally been thought to be important for initiating general arousal states. This is in part due to initial reports of changes in unit activity during transitions from sleep to wakefulness (Huttenlocher 1961; Kasamatsu 1970; Manohar et al. 1972). In addition, a more specific role for the reticular formation in attention has been described in primates performing visual discrimination tasks (Pragay et al. 1978; Fabre et al. 1983). This is consistent with reports of sensory neglect following reticular formation lesions (Watson et al. 1974). Together, these foundational data suggest that reticular formation may function to enhance the overall level of arousal and vigilance necessary for attending to and acting upon salient stimuli (Mesulam 1981). Accordingly, changes in reward-related MRNm neuronal activity could provide an important signal indicating that the contingencies of recently executed behaviors have changed.The striking similarity of the reward prediction error signals of MRNm neurons reported here suggests that MRNm, along with brain regions such as lateral habenula (Matsumoto and Hikosaka 2007), pedunculopontine nucleus (PPTg) (Kobayashi and Okada 2007), and dorsal raphé nucleus (Nakamura et al. 2008) may contribute to the generation of reward prediction error signals observed in dopamine neurons. Furthermore, these data suggest the possibility that such signals are a general property of a large network of midbrain structures. Given that the projections from the reticular formation to VTA are glutamatergic (Geisler et al. 2007), it is possible that the changes in reward-related activity of MRNm neurons, in concert with PPTg, could provide an excitatory component of the reward prediction error signal. In combination with inhibitory inputs from lateral habenula, this may then selectively activate dopamine neurons to initiate the coordinated selection of appropriate behaviors in response to changes in reward outcome (Humphries et al. 2007).     

Bias in self-evaluation: Signal probability effects

Two experiments examined apparent signal probability effects in simple verbal self-reports. After each trial of a delayed matching-to-sample task, young adults pressed either a “yes” or a “no” button to answer a computer-presented query about whether the most recent choice met a point contingency requiring both speed and accuracy. A successful matching-to-sample choice served as the “signal” in a signal-detection analysis of self-reports. Difficulty of matching to sample, and thus signal probability, was manipulated via the number of nonmatching sample and comparison stimuli. In Experiment 1, subjects exhibited a bias (log b) for reporting matching-to-sample success when success was frequent, and no bias or a bias for reporting failure when success was infrequent. Contingencies involving equal conditional probabilities of point consequences for “I succeeded” and “I failed” reports had no systematic effect on this pattern. Experiment 2 found signal probability effects to be evident regardless of whether referent-response difficulty was manipulated in different conditions or within sessions. These findings indicate that apparent signal probability effects in self-report bias that were observed in previous studies probably were not an artifact of contingencies intended to improve self-report accuracy or of the means of manipulating signal probability. The findings support an analogy between simple self-reports and psychophysical judgments and bolster the conclusion of Critchfield (1993) that signal probability effects can influence simple self-reports much as they do reports about external stimuli in psychophysical experiments.     

The Organization of Extrinsic Neurons and Their Implications in the Functional Roles of the Mushroom Bodies in Drosophila melanogaster Meigen

Although the importance of the Drosophila mushroom body in olfactory learning and memory has been stressed, virtually nothing is known about the brain regions to which it is connected. Using Golgi and GAL4–UAS techniques, we performed the first systematic attempt to reveal the anatomy of its extrinsic neurons. A novel presynaptic reporter construct, UAS-neuronal synaptobrevin–green fluorescent protein (n-syb–GFP), was used to reveal the direction of information in the GAL4-labeled neurons. Our results showed that the main target of the output neurons from the mushroom body lobes is the anterior part of the inferior medial, superior medial, and superior lateral protocerebrum. The lobes also receive afferents from these neuropils. The lack of major output projections directly to the deutocerebrum’s premotor pathways discourages the view that the role of the mushroom body may be that of an immediate modifier of behavior. Our data, as well as a critical evaluation of the literature, suggest that the mushroom body may not by itself be a “center” for learning and memory, but that it can equally be considered as a preprocessor of olfactory signals en route to “higher” protocerebral regions.     

Reinforcement omission on temporal go-no-go schedules

Either a partial blackout, or the blackout plus a “feeder flash”, occurred in lieu of reinforcement on two procedures that produced opposite patterns of responding after reinforcement. Response rate was elevated after reinforcement omission on the procedure that produced a “pause-and-respond” pattern following reinforcement, but depressed after reinforcement omission on the procedure that produced a “respond-and-pause” pattern. The effect of blackout plus feeder flash was generally intermediate between the effects of blackout and the effects of reinforcement. These results are consistent with an interpretation of reinforcement omission effects in terms of the discriminative temporal control exerted by reinforcement and stimuli similar to it.     

Punishment: the interactive effects of delay and intensity of shock

A discrete-trial punishment procedure, with rats, was used to examine how delay-of-shock intervals of 0 to 28 sec and shock intensity interact to decrease the frequency and increase the latency of a positively reinforced response. For delay-of-shock intervals of 0, 7, 14, and 28 sec, there was a range of shock intensities, for some subjects, over which the punishing effect of shock was an increasing, monotonic function of shock intensity. For other subjects this transition was abrupt. Functions relating response frequency and latency measures to shock intensity were displaced toward higher values on the shock intensity axis with an increase in delay-of-shock interval. The effects of “gradual” and “abrupt” introduction to “severe” shock, as well as re-exposure to previously used shock intensities, were examined under both the immediate and delay-of-shock conditions. With delay-of-shock intervals of 7, 14, or 28 sec, shock intensities of approximately 0.50 milliamperes or greater were necessary to decrease substantially the number and increase the latency of the lever-pressing response. For the immediate punishment group this intensity was approximately 0.20 ma. These facts were related to Annau and Kamin's (1961) conditioned emotional response experiment in which a shock intensity of 0.49 ma or greater was required to suppress the rate of a positively reinforced response.     

Stimulus properties of conspecific behavior

Two experiments identified the conditions in which the behavior of one bird acquired discriminative control of the behavior of a second bird. The schedule-controlled behaviors of the “stimulus” bird were differentially correlated with the components of a multiple schedule according to which the pecking of an “experimental” bird produced food. In Experiment 1, three pairs of pigeons acquired a successive discrimination and two reversals with the conspecific stimuli. Experiment 2 included a control condition in which no systematic relationship existed between the conspecific stimuli and the component schedules. While differential responding during the components of the multiple schedule was again found when the conspecific stimuli were available, differential responding did not occur in the control condition. Test conditions included in the experiments indicated that (a) the differential responding was not dependent on the discriminative properties of reinforcement, (b) the pecking of the stimulus and experimental birds was temporally interrelated, (c) the visual conspecific stimuli were critical to the maintenance of the discrimination, and (d) the observed stimulus control immediately generalized to an unfamiliar conspecific.     

Development of a complex multiple schedule in the chimpanzee

The development of chimpanzee behavior on a four-component, three-lever multiple schedule is described. Component schedules included the Sidman avoidance procedure with a concurrent discriminated avoidance schedule on a second lever, fixed ratio performance for food, differential reinforcement of low rate for water requiring a dual response chain, and a symbol discrimination task for continuous food reinforcement using three levers. The avoidance component of this schedule was employed during the January 31, 1961 suborbital space flight of the chimpanzee “Ham.” On November 29, 1961, the chimpanzee “Enos” performed on the multiple schedule during three orbits around the earth in a Mercury capsule.     

Sample-specific ratio effects in matching to sample

In a symbolic matching-to-sample task, pigeons were trained using sample-specific, fixed-ratio “observing responses.” Subsequently, in a mixed condition, each sample was presented equally often with each ratio requirement, i.e., the ratios were no longer correlated with the samples. In a second experiment, pigeons were trained initially in the mixed condition and subsequently shifted to the sample-specific condition in which the required ratios were correlated with the samples. Results of both experiments suggested joint control of choices by ratio value and by the exteroceptive stimuli. The discriminative properties of the ratios appeared to outweigh absolute ratio-size effects.     

Memory for emotionally provocative words in alexithymia: A role for stimulus relevance

Alexithymia is associated with emotion processing deficits, particularly for negative emotional information. However, also common are a high prevalence of somatic symptoms and the perception of somatic sensations as distressing. Although little research has yet been conducted on memory in alexithymia, we hypothesized a paradoxical effect of alexithymia on memory. Specifically, recall of negative emotional words was expected to be reduced in alexithymia, while memory for illness words was expected to be enhanced in alexithymia.Eighty-five high or low alexithymia participants viewed and rated arousing illness-related (“pain”), emotionally positive (“thrill”), negative (“hatred”), and neutral words (“horse”). Recall was assessed 45 min later.High alexithymia participants recalled significantly fewer negative emotion words but also more illness-related words than low alexithymia participants. The results suggest that personal relevance can shape cognitive processing of stimuli, even to enhance retention of a subclass of stimuli whose retention is generally impaired in alexithymia.     

Opposing effects on muscarinic acetylcholine receptors in the piriform cortex of odor-trained rats

We combined pharmacological studies and electrophysiological recordings to investigate modifications in muscarinic acetylcholine (ACh) receptors (mAChR) in the rat olfactory (piriform) cortex, following odor-discrimination rule learning. Rats were trained to discriminate between positive and negative cues in pairs of odors, until they reached a phase of high capability to learn unfamiliar odors, using the same paradigm (“rule learning”). It has been reported that at 1–3 d after the acquisition of odor-discrimination rule learning, pyramidal neurons in the rat piriform cortex show enhanced excitability, due to a reduction in the spike-activated potassium current I_AHP, which is modulated by ACh. Further, ACh and its analog, carbachol (CCh), lost the ability to reduce the I_AHP in neurons from trained rats. Here we show that the reduced sensitivity to CCh in the piriform cortex results from a decrease in the number of mAChRs, as well as a reduction in the affinity of the receptors to CCh. Also, it has been reported that 3–8 d after the acquisition of odor-discrimination rule learning, synaptic transmission in the piriform cortex is enhanced, and paired-pulse facilitation (PPF) in response to twin stimulations is reduced. Here, intracellular recordings from pyramidal neurons show that CCh increases PPF in the piriform cortex from odor-trained rats more than in control rats, suggesting enhanced effect of ACh in inhibiting presynaptic glutamate release after odor training.     

Avoidance of risk as a determinant of cooperation

Pairs of subjects could either cooperate or respond on a lower paying individual task. Whenever both subjects chose to cooperate, either subject could make a response that took $1.00 of the other's earnings. In Exp. I, a stimulus signalled when a “take” response had been made. Either subject could avoid the loss by switching to the individual task within 5 sec after the stimulus appeared. Rates of cooperation were high when losses could be avoided but decreased again when the avoidance condition was removed. In Exp. II, a response prevented “takes” from occurring for a specified time interval after the response. This procedure also maintained cooperation. When each avoidance response subtracted from earnings, both avoidance responding and cooperation were eliminated.     

Pigeons can discriminate locations presented in pictures

The present experiments were designed to teach pigeons to discriminate two locations represented by color photographs. Two sets of photographs were taken at two distinctive locations on a university campus. These sets represented several standpoints at each location. For the true-discrimination group, pictures from the two locations were differentially associated with reward; for the pseudodiscrimination group, half of the views from each location were arbitrarily but consistently associated with reward. The former group acquired the discrimination much more rapidly. These birds also showed good transfer to new views from the standpoints used in training and to a new standpoint at each location not used in training. In a second experiment, another group of pigeons could terminate any training trial by pecking an “advance” key. Three of 4 subjects used this option to reduce the duration of trials in which pictures from the negative location were presented. These data suggest that pigeons can integrate views shown in pictures into a “concept” of a location. The method used here may be the experimental analogue of a common, natural process by which animals learn to identify locations.