Sampling and decision rules used by honey bees in a foraging arena

Animals must continuously choose among various available options to exploit the most profitable resource. They also need to keep themselves updated about the values of all available options, since their relative values can change quickly due to depletion or exploitation by competitors. While the sampling and decision rules by which foragers profitably exploit a flower patch have attracted a great deal of attention in theory and experiments with bumble bees, similar rules for honey bee foragers, which face similar foraging challenges, are not as well studied. By presenting foragers of the honey bee Apis cerana with choice tests in a foraging arena and recording their behavior, we investigate possible sampling and decision rules that the foragers use to choose one option over another and to track other options. We show that a large part of the sampling and decision-making process of a foraging honey bee can be explained by decomposing the choice behavior into dichotomous decision points and incorporating the cost of sampling. The results suggest that a honey bee forager, by using a few simple rules as part of a Bayesian inference process, is able to effectively deal with the complex task of successfully exploiting foraging patches that consist of dynamic and multiple options.     

The ambiguous-cue problem: The effects of reduced similarity between trials and the schedule of reinforcement

In a series of seven experiments pigeons were trained on the two-choice ambiguous-cue problem that contains PA and NA trials. On PA trials choice of P (the positive stimulus) was reinforced and choice of A (the so-called ambiguous stimulus) was not reinforced; on NA trials choice of A was reinforced and choice of N (the negative stimulus) was not reinforced. When the two types of trials were distinguished by only the presence of P or N and each trial was terminated by one response, subjects quickly attained high levels of performance on NA trials but showed only chance levels of performance on PA trials. PA performance was markedly improved when a variety of additional stimuli distinguished the two types of trials and when a variety of schedules of intermittent reinforcement were employed; these manipulations had little effect on NA performance.     

Resurgence of derived stimulus relations.

Resurgence has been shown in human and nonhuman operant behavior, but not in derived relational responses. The present study examined this issue. Twenty-three undergraduates were trained to make conditional discriminations in a three-choice matching-to-sample paradigm. The training resulted in three equivalence classes, each consisting of four arbitrarily configured visual stimuli. The same 12 stimuli were then reorganized, and the conditional discrimination training was repeated such that three new classes were possible. In a subsequent test of derived relations, most subjects showed response patterns that were consistent with the altered conditional discriminations. Subjects were then exposed to conditional discrimination trials under extinction. Most subjects continued to respond consistently with the most recently reinforced conditional discrimination trials. During the next phase, subjects were exposed to symmetry and equivalence trials. Responses consistent with the most recent training produced feedback saying that the responses were incorrect, whereas other responses produced no feedback. Most subjects showed a resurgence of responding that was consistent with their earlier training. Finally, subjects were exposed to conditional discrimination trials carried out in extinction. Most subjects continued to show a resurgence of responding that was consistent with their early training.     

Stimulus control in the goldfish after massed extinction

Compared with the data of goldfish trained only with stimuli correlated with reinforcement, interspersed reinforcement-stimulus and extinction-stimulus trials resulted in sharper stimulus control and a marked reduction in the percentage of key-presses emitted in the presence of stimuli located near the extinction stimulus on the test dimension. If non-reinforced trials were not interspersed with reinforced trials, there was no sharpening of stimulus control and less reduction in key presses in the presence of stimuli near the extinction stimulus on the test dimension.     

Do inexperienced bumblebee foragers use scent marks as social information?

Bumblebees (Bombus spp.) foraging in the field typically reject flowers where they detect the olfactory footprints of previous visitors and hence avoid recently emptied inflorescences. A growing number of studies have begun to illustrate that associative learning shapes the development of this process, in both bumblebees and other bee species. This raises the question of what the default response to such marks is, but little is known about how inexperienced foragers use social information. Here, we offered flower-naive bees a choice between scent-marked flowers and unmarked alternatives and found that individuals neither avoided nor preferred marked flowers. Our findings provide no support for ‘hard-wired’ responses to scent marks in bumblebees and highlight the importance of associative learning in shaping social information use to match local circumstances.     

Detecting a nonevent: Delayed presence-versus-absence discrimination in pigeons

Eight pigeons were trained on a delayed presence-versus-absence discrimination paradigm in which a sample stimulus was presented on some trials but not on others. If a sample was presented, then a response to one choice key produced food. If no sample was presented, a response to the other choice key produced food. The basic finding was that performance remained constant and well above 50% correct on no-sample trials as the retention interval increased, whereas performance dropped precipitously (to below 50% correct) on sample trials. In the second phase of the experiment, all of the trials were no-sample trials, and reinforcers were delivered probabilistically for one group of pigeons and according to time-based schedules for the other group. The exact reinforcement probabilities used in Phase 2 were those calculated to be in effect on no-sample trials in Phase 1 (according to a discrete-state model of performance). Subjects did not show exclusive preference for the richer alternative on no-sample trials in the first phase, but those in the probabilistic group developed near-exclusive preference for the richer alternative during the second phase. These data are inconsistent with the predictions of the discrete-state model, but are easily accommodated by an account based on signal detection theory, which also can be applied effectively to discrimination of event duration and the “subjective shortening” effect.     

The social control of generalized imitation

Instructions, discrimination procedures, and sources of reinforcement were manipulated in order to determine the bases for the maintained "non-reinforced" imitations observed in generalized imitation research. Six girls received imitation training from two experimenters. One experimenter modelled only reinforced responses; the other modelled only non-reinforced responses. The children imitated all responses when no reinforced alternative was available, even though results of choice procedures and special instructions clearly demonstrated that they discriminated reinforced from non-reinforced responses. Instructions not to perform non-reinforced imitations immediately eliminated these behaviors. It is suggested that social setting events may be largely responsible for generalized imitation.     

Spatial relational learning in rufous hummingbirds (Selasphorus rufus)

There is increasing evidence that animals can learn abstract spatial relationships, and successfully transfer this knowledge to novel situations. In this study, rufous hummingbirds (Selasphorus rufus) were trained to feed from either the lower or the higher of two flowers. When presented with a test pair of flowers, one of which was at a novel height, they chose the flower in the appropriate spatial position rather than the flower at the correct height. This response may also have been influenced by a preference for taller flowers as acquisition of the task during experimental training occurred more readily when the reward flower was the taller of the pair. Thus, it appears that although learning abstract relationships may be a general phenomenon across contexts, and perhaps across species, the ease with which they are learned and the context in which they are subsequently used may not be the same.     

Emergent Simple Discrimination via Transfer from Differentially Reinforced S+ Stimuli: A Further Test of the Stimulus-Response Interaction Model

This study examined emergent form discrimination through contiguity of the forms (B stimuli) with colours (A stimuli) associated with different magnitudes of reinforcement. In Experiment 1, children were trained on two colour discriminations. In both, the colour stimuli had the same form: A1B1/A2B1 and A3B1/A2B1. Responding to A1/B1 was followed by three units of reinforcement, to A3B1 by one unit of reinforcement, and to A2B1 by no reinforcement. There followed five tests in which stimuli were presented without programmed consequences. These assessed: (1) the maintenance of accurate performance on the training tasks; (2) S+ preference (A1B1/A3B1); (3) stimulus generalization (A1B2/A2B3, A3B4/A2B5); (4) S+ preference again (A1B6/A3B6); and (5) transfer from the S+ colours to the forms (A4B2/A4B4). On the final test, seven of the eight subjects selected A4B2. Experiment 2 demonstrated that a preference for A4B2 also occurred when the S+ preference tests were omitted. This preference was also found when the total amounts of reinforcement associated with the S+ colours (A1 and A3) were the same (Experiment 3). These findings: (a) indicate that children can acquire new discriminations under non-reinforced conditions through contiguity with differentially reinforced S+ stimuli; (b) support the Stimulus-Response Interaction model (Smeets, 1993) for specifying transfer.     

Overcoming unlearned response biases: delayed escape following errors facilitates acquisition of win-stay and win-shift working memory water-escape tasks in rats

Previous research has demonstrated that perseveration in escape situations is an unlearned response bias of rats and mice which is difficult to overcome. In Experiment I, Sprague-Dawley rats were trained to a criterion of nine correct choices in 10 consecutive trials in a win-stay working memory water-escape task wherein an escape platform was hidden in one of two compartments. Subjects were given a forced choice on an Information Trial followed by a free choice on a Test Trial 5 min later. Subjects who were given a 30-s forced swim in the incorrect section following errors on the Test Trial showed faster acquisition and less use of response perseveration than control subjects. In Experiment II, the delayed escape procedure was used to compare acquisition of win-stay and win-shift strategies. Contrary to previous research both groups learned the tasks, although the win-stay group showed better performance. It is concluded that unlearned response biases of perseverating and returning to previous escape sites can be overcome if experimental conditions are appropriately arranged.     

Preference for unpredictable food rewards occurs with high proportion of reinforced trials or alcohol if rewards are not delayed

Organisms typically prefer situations where reward and nonreward are predictable rather than unpredictable. Although many theories can account for this result (e.g., information theory and delay-reduction theory), a recently developed mathematical model (DMOD) also predicts that subjects prefer the unpredictable reward situation under conditions that substantially decrease aversiveness of unpredictable nonreward (Daly & Daly, 1982). Because a high proportion of reinforced trials (lenient schedule) and alcohol injections decrease aversive conditioning, these variables were tested with rats in five E-maze experiments. A choice to one side of the maze resulted in a stimulus uncorrelated with reward outcome (unpredictable situation). A choice to the other side resulted in stimuli correlated with reward and nonreward (predictable situation). The stimuli were not visible until after the choice was made. A lenient reinforcement schedule resulted in preference for the unpredictable reward situation if rewards were not delayed. Alcohol resulted in preference for the unpredictable reward situation if a medium five-pellet reward was given. A lenient reinforcement schedule combined with an alcohol injection resulted in faster acquisition of the preference for the unpredictable reward situation than did a lenient schedule combined with a saline control injection. These results pose a major challenge to most theories, yet were predicted by DMOD.     

Representation of serial order in pigeons (Columba livia)

In Experiment 1, 2 groups of pigeons were trained to respond to either a 4-item (A→B→C→D) or 5-item (A→B→C→D→E) list. After learning their respective list, half of the subjects were trained on a positive pair with reinforcement provided when pairs were responded to in the order true to that of the original sequence (4-item: B→C; 5-item: B→D). The remaining subjects were trained on a negative pair with reinforcement provided for responding to the pairs in the order opposite to that learned in the original sequence (4-item: C→B; 5-item: D→B). Subjects in the positive pair condition learned their respective pair faster than did subjects in the negative pair condition. In Experiment 2, after reaching criterion on a 4-item list, subjects received 16 BC probe trials spread across 4 sessions of training. Subjects performed significantly above chance on the probe trials. The performance of our subjects in Experiments 1 and 2 demonstrates that, similar to monkeys, pigeons form a representation of the lists that they learn.     

Emergent Simple Discriminations and Conditional Relations in Children, Intellectually Impaired Adults,

Previous research has shown that when, under non-reinforced conditions, stimuli are added to S+ and S- stimuli in simultaneous discrimination tasks, transfer between paired stimuli is likely to occur. The present study examined whether this procedure also leads to the formation of conditional relations between paired stimuli. In Experiments 1, 2, and 3, normal pre-school children were trained on a simultaneous discrimination task with A1 reinforced and A2 not reinforced (A1+/A2- ). Then they received two tests (no programmed consequences): one with B stimuli superimposed on the A stimuli (A1B1/ A2B2), and one with B stimuli only (B1/B2). Subjects who selected A1B1 and B1 also received conditional discrimination tests: one with B1 or B2 as samples and A1 and A2 as comparisons (B-A), and one in which the functions of these stimuli were reversed (A-B). Intellectually impaired adults and normal adults served in Experiments 4 and 5, respectively. These experiments were basically the same except that the subjects were also given the opportunity to demonstrate transfer from B to C via BC (B1C1/B2C2 and C1/C2 tests). Most children (75%) and intellectually impaired adults (75%) treated the conditional discrimination probe tasks as simple discriminations and typically selected the trained and derived S+ stimuli. The remaining children, intellectually impaired adults, and all normal adults related all directly and indirectly linked stimuli of the same functions conditionally to one another (A-B, B-C, A-C, and vice versa). The present findings suggest that, as humans develop, conditional stimulus relations may emerge from tasks and stimulus configurations that are increasingly remote from traditional conditional discrimination tasks.     

Effects on Sexual Conditioning of Devaluing the US through Satiation

Male Japanese quail were conditioned to approach a visual CS that predicted an opportunity to copulate with a female quail. Subsequently the subjects were tested in both non-reinforced and reinforced test trials while sexually satiated and sexually deprived. In the satiation condition the sexual US was devalued by allowing the male subjects to copulate repeatedly with female birds just prior to testing; in the deprivation condition, the males were not allowed to copulate for a day prior to testing. The subjects displayed significantly less approach to the CS when sexually satiated than when sexually deprived. These results are compatible with a mechanism in which performance of Pavlovian sexual approach conditioning is mediated by activation of a memorial representation of the US.     

Theories of probabilistic reinforcement.

In three experiments, pigeons chose between two alternatives that differed in the probability of reinforcement and the delay to reinforcement. A peck at a red key led to a delay of 5 s and then a possible reinforcer. A peck at a green key led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials so as to estimate an indifference point, or a duration at which the two alternatives were chosen about equally often. In Experiments 1 and 2, the intertrial interval was varied across conditions, and these variations had no systematic effects on choice. In Experiment 3, the stimuli that followed a choice of the red key differed across conditions. In some conditions, a red houselight was presented for 5 s after each choice of the red key. In other conditions, the red houselight was present on reinforced trials but not on nonreinforced trials. Subjects exhibited greater preference for the red key in the latter case. The results were used to evaluate four different theories of probabilistic reinforcement. The results were most consistent with the view that the value or effectiveness of a probabilistic reinforcer is determined by the total time per reinforcer spent in the presence of stimuli associated with the probabilistic alternative. According to this view, probabilistic reinforcers are analogous to reinforcers that are delivered after variable delays.     

Foraging behaviour in domestic pigs (Sus scrofa): remembering and prioritizing food sites of different value

This experiment investigated whether domestic pigs can remember the locations of food sites of different relative value, and how a restricted retrieval choice affects their foraging behaviour. Nine juvenile female pigs were trained to relocate two food sites out of a possible eight in a spatial memory task. The two baited sites contained different amounts of food and an obstacle was added to the smaller amount to increase handling time. On each trial, a pig searched for the two baited sites (search visit). Once it had found and eaten the bait, it returned for a second (relocation) visit, in which the two same sites were baited. Baited sites were changed between trials. All subjects learnt the task. When allowed to retrieve both baits, the subjects showed no preference for retrieving a particular one first (experiment 1). When they were allowed to retrieve only one bait, a significant overall preference for retrieving the larger amount emerged across subjects (experiment 2). To test whether this preference reflected an avoidance of the obstacle with the smaller bait, 15 choice-restricted control trials were conducted. In control trials obstacles were present with both baits. Pigs continued to retrieve the larger bait, indicating they had discriminated between the two food sites on the basis of quantity or profitability and adjusted their behaviour accordingly when the relocation choice was restricted. This suggests for the first time that domestic pigs have the ability to discriminate between food sites of different relative value and to remember their respective locations.     

Probability matching: encouraging optimal responding in humans.

Two hundred undergraduate students participated in a repeated-trials binary choice procedure in which choice of one outcome was correct on 75% of trials. Subjects received 192 trials and were divided into five conditions: (1) control; (2) subjects were given the actual probabilities; (3) subjects were told if they did well they could leave early; (4) competition condition; (5) midway through the task subjects were asked to recommend a strategy for another subject. Half of the subjects in each group were told that the best they could do was to be correct on 75% of the trials. This manipulation permitted assessment of the hypothesis that subjects in probability-matching tasks are seeking a strategy that will be correct on 100% of the trials. The results partially confirmed this hypothesis. In addition, two of the variables improved performance significantly (giving probabilities and asking subjects to recommend a strategy). However, while subjects in all groups improved significantly over trials, optimal choice did not occur in this task.     

Effect of subvocalization on memory for speech sounds.

Subjects in two groups were presented with sequences of 6 consonant-vowel syllables for ordered recall. Subjects in both groups were trained to suppress subvocalization. Only subjects in the feedback group were asked to suppress subvocalization during the experiment, while subjects in the no-feedback group were allowed to subvocalize during the memory task. Analysis of the electromyographic records showed a large decrease in subvocalization in the feedback condition, and results of the memory task revealed an increase in errors for this group. However, a comparison of errors within each group revealed an identical pattern of overall errors and intrusion errors on subvocalization and nonsubvocalization trials, and these results were identical in both groups. The results suggest that requiring subjects to simultaneously suppress subvocalization and remember syllables depresses performance slightly, but encoding of speech sounds in short-term memory occurs independently of subvocal activity during the memory task.     

Peak shift in honey bee olfactory learning

If animals are trained with two similar stimuli such that one is rewarding (S+) and one punishing (S?), then following training animals show a greatest preference not for the S+, but for a novel stimulus that is slightly more different from the S? than the S+ is. This peak shift phenomenon has been widely reported for vertebrates and has recently been demonstrated for bumblebees and honey bees. To explore the nature of peak shift in invertebrates further, here we examined the properties of peak shift in honey bees trained in a free-flight olfactory learning assay. Hexanal and heptanol were mixed in different ratios to create a continuum of odour stimuli. Bees were trained to artificial flowers such that one odour mixture was rewarded with 2 molar sucrose (S+), and one punished with distasteful quinine (S?). After training, bees were given a non-rewarded preference test with five different mixtures of hexanal and heptanol. Following training bees’ maximal preference was for an odour mixture slightly more distinct from the S? than the trained S+. This effect was not seen if bees were initially trained with two distinct odours, replicating the classic features of peak shift reported for vertebrates. We propose a conceptual model of how peak shift might occur in honey bees. We argue that peak shift does not require any higher level of processing than the known olfactory learning circuitry of the bee brain and suggest that peak shift is a very general feature of discrimination learning.     

Social control by adult preference in operant conditioning with children

During daily sessions four institutionalized retardates and four normal, first-grade children each played a two-choice marble dropping game while an adult experimenter sat beside and watched. When one of two colored lights was presented, two tokens were dispensed as soon as the subject dropped a marble, and when the other light was on, one token was dispensed. During choice trials that were interspersed among single-colored trials the subject was required to choose one of the two different colored holes. After the subject developed a preference for the color that earned two tokens, the experimenter stated a preference for the response that earned only one token (i.e., “I like it better when you put it into the —colored hole.”) Six of the eight subjects immediately switched to the response that earned low magnitude reinforcement when the experimenter stated his preference. Only two subjects switched back to the response that earned high magnitude reinforcement when the experimenter left the room. Subjects complied with the experimenter's preference statement regardless of whether or not it also included a negative component (i.e., what the experimenter did not like). There were no systematic differences between mentally retarded and normal subjects.