Second-order contrast based on the expectation of effort and reinforcement

Pigeons prefer signals for reinforcement that require greater effort (or time) to obtain over those that require less effort to obtain (T. S. Clement, J. Feltus, D. H. Kaiser, & T. R. Zentall, 2000). Preference was attributed to contrast (or to the relatively greater improvement in conditions) produced by the appearance of the signal when it was preceded by greater effort. In Experiment 1, the authors of the present study demonstrated that the expectation of greater effort was sufficient to produce such a preference (a second-order contrast effect). In Experiments 2 and 3, low versus high probability of reinforcement was substituted for high versus low effort, respectively, with similar results. In Experiment 3, the authors found that the stimulus preference could be attributed to positive contrast (when the discriminative stimuli represented an improvement in the probability of reinforcement) and perhaps also negative contrast (when the discriminative stimuli represented reduction in the probability of reinforcement).     

Rats place greater value on rewards produced by high effort: An animal analogue of the “effort justification” effect

The effort justification phenomenon, in which greater value is given to rewards that require more effort to obtain, is frequently explained in terms of cognitive dissonance (Festinger & Carlsmith, 1959). Here, we employed a novel combination of operant and lick analysis techniques to show that rats place more value on a sucrose reward when it follows high effort than when the same reward follows low effort. This is the first demonstration of a direct analogue of the effort justification phenomenon in nonhuman animals. As the behavior of rats is normally considered in terms of relatively simple mechanisms, the current results question the need for complex cognitive accounts for the effort justification phenomenon (in rats or humans). As an alternative, we examine the possibility that high effort produces an aversive state that enhances reward value by a process of contrast.     

The effect of an aversive event on the return of fear

An experiment on the return-of-fear (ROF) was carried out on 40 snake- or spider-phobic subjects in order to determine whether an arousing event that occurs shortly before retest influences the magnitude of the ROF. Additionally, we attempted to produce a reinstatement of fear by introducing an unrelated aversive event (shock) after the fear had been reduced. The arousal manipulation successfully increased subjective arousal but not heartrate responsiveness. The increases in subjective arousal were not followed by increases in ROF, and the attempt to produce a reinstatement of fear did not succeed. A significant correlation between speed of fear-reduction and ROF was obtained.     

Effects of cocaine on briefly signaled versus completely signaled delays to reinforcement.

Key pecking by 4 pigeons was maintained by a multiple schedule consisting of two variable-interval 60-s schedules wherein each food presentation followed a nonresetting 27-s delay that was either briefly signaled at its outset or completely signaled. Brief-signal duration was adjusted so that response rates maintained by the briefly and completely signaled delays of reinforcement were similar. In general, acute administration of small to intermediate doses (0.3 to 3.0 mg/kg) of cocaine produced either small increases in response rates in both components or no change, and larger doses (5.6 to 13.0 mg/kg) decreased response rates. Chronic (i.e., daily) cocaine administration (10.0 mg/kg) resulted in tolerance to the rate-decreasing effects in both components. Cocaine's effects were generally similar whether delays were completely or briefly signaled. Discontinuation of cocaine administration and subsequent removal of the delay signals also had similar effects in both components of the multiple schedule. Taken together, these results are consistent with the view that the two types of delay signals were equally effective in maintaining responding during the variable-interval schedules.     

Audiovisual phenomenal causality

We report three experiments in which visual or audiovisual displays depicted a surface (target) set into motion shortly after one or more events occurred. A visual motion was used as an initial event, followed directly either by the target motion or by one of three marker events: a collision sound, a blink of the target stimulus, or the blink together with the sound. The delay between the initial event and the onset of the target motion was varied systematically. The subjects had to rate the degree of perceived causality between these events. The results of the first experiment showed a systematic decline of causality judgments with an increasing time delay. Causality judgments increased when additional auditory or visual information marked the onset of the target motion. Visual blinks of the target and auditory clacks produced similar causality judgments. The second experiment tested several models of audiovisual causal processing by varying the position of the sound within the visual delay period. No systematic effect of the sound position occurred. The third experiment showed a subjective shortening of delays filled by a clack sound, as compared with unfilled delays. However, this shortening cannot fully explain the increased tolerance for delays containing the clack sound. Taken together, the results are consistent with the interpretation that the main source of the causality judgments in our experiments is the impression of a plausible unitary event and that perfect synchrony is not necessary in this case.     

Evaluating the effect of delay spacing on delay discounting: carry‐over effects on steepness and the form of the discounting function

Most delay discounting studies use tasks that arrange delay progressions in which the spacing between consecutive delays becomes progressively larger. To date, little research has examined delay discounting using other progressions. The present study assessed whether the form or steepness of discounting varied across different delay progressions. Human participants completed three discounting tasks with delay progressions that varied in the time between consecutive delays: a standard (increasing duration between delays), linear (equal duration between delays), and an inverse progression (decreasing duration between delays). Steepness of discounting was generally reduced, and remained so, following experience with the inverse progression. Effects of the delay progression on the best fitting equation were order‐dependent. Overall the hyperbola model provided better fits, but the exponential model performed better with data from the inverse progression. Regardless, differences in which model fit best were often small. The finding that the best fitting model was dependent, in some cases, on the delay progression suggests that a single quantitative model of discounting may not be applicable to describe discounting across all procedural contexts. Ultimately, changes in steepness of discounting following experience with the inverse progression appeared similar to anchoring effects, whose mechanism will require further study to delineate.     

Delayed alternation in the pigeon

Pigeons were studied in a delayed-response task requiring alternation of key pecks on two response keys. Blackouts of from 1 to 10 sec intervened between successive choices on the two keys.

The following results were obtained: (1) Birds performed at well above chance accuracy on all the delays tested. Accuracy was generally lowest at 1- and 10-sec delays. (2) Overt postural orientations during the delay interval appeared to mediate accurate key-pecking behavior. (3) The shape of the delay vs. accuracy function was discussed in terms of the possibly confounding influences of (a) stimulus “trace” variables, and (b) aversive effects of the time outs produced by incorrect responding.

     

The problem of delayed causation in a video game: Constant, varied, and filled delays

A first-person shooter video game was adapted for the study of causal decision making within dynamic environments. The video game included groups of three potential targets. Participants chose which of the three targets in each group was producing distal explosions. The actual source of the explosion effect varied in the delay between the firing of its weapon and the effect (from 0 to 2 s), whether these programed average delays were constant or varied from shot to shot, and whether the delays were unfilled or filled with an auditory event. In Experiment 1, participants’ choice accuracy was highest with shorter delays, but there was no effect of filling the delay and some beneficial effect of varying the delay. These results were re-examined in Experiment 2 but with participants experiencing the same average delay for seven subsequent decisions before the next average delay was introduced. In this experiment, men showed a strong and consistent benefit of filling a delay whereas women did not. Participants’ behavior is considered within the context of a model that assumes that choice behavior is driven by experienced contiguity for the target and foils.     

Attempts to prime intellectual virtues for understanding of science: Failures to inspire intellectual effort

Strategies for effectively communicating scientific findings to the public are an important and growing area of study. Recognizing that some complex subjects require recipients of information to take a more active role in constructing an understanding, we sought to determine whether it was possible to increase subjects’ intellectual effort via “priming” methodologies. In particular, we asked whether subconsciously priming “intellectual virtues” (IVs), such as curiosity, perseverance, patience, and diligence might improve participants’ effort and performance on various cognitive tasks. In the first experiment, we found no significant differences in either effort or understanding between IV-primed and neutrally-primed individuals across two different priming techniques. The second experiment measured the effect of IV-priming on intellectual effort in simpler, shorter-duration puzzles and exploration activities; here, we observed an effect, but given its low strength and short duration, we do not believe that priming of IVs is a promising strategy for science communication.     

When the event is more than the sum of its parts: 9-month-olds' long-term ordered recall

Research suggests that 9-month-old infants are able to recall single object-specific actions over delays of 24 hours. In the present research we investigated whether 9-month-olds are able to recall over more extended delays, and to recall the temporal order of events, as well as the individual actions in them. In addition, we investigated whether recall can be enhanced by pre- and/or re-exposure to target events. Using elicited imitation of novel, multi-step event sequences, we demonstrated that, as a group, 9-month-olds are able to recall target actions after delays of five weeks. However, after this long delay, only 45% of the infants recalled the temporal order of the events. Re-exposure to events during the delay interval proved necessary for boys, but not for girls; pre-exposure to events did not affect later recall. The implications of individual differences in infants' recall ability for the understanding of the development of the neural correlates of declarative memory are discussed.     

Thinking about the future early in life: The role of relational memory

The constructive episodic simulation hypothesis suggests that we imagine possible future events by flexibly recombining details of past experiences to produce novel scenarios. Here we tested this hypothesis by determining whether episodic future thinking is related to relational memory ability during the preschool years. Children (3- to 5-year-olds) were asked to remember a past event and imagine a possible future event using an adapted version of the recombination paradigm. Relational learning and inference were assessed using a task adapted from the neuroimaging literature. The results show that preschoolers were able to describe both past and possible future events; however, they produced more specific episodic details in relation to past events relative to future events. Episodic future thinking performance was correlated with performance on the relational inference task, consistent with the idea that the ability to flexibly recombine relational knowledge is critical in episodic future thinking.     

Disruption of responding maintained by conditioned reinforcement: alterations in response-conditioned-reinforcer relations

An observing procedure was used to investigate the effects of alterations in response-conditioned-reinforcer relations on observing. Pigeons responded to produce schedule-correlated stimuli paired with the availability of food or extinction. The contingency between observing responses and conditioned reinforcement was altered in three experiments. In Experiment 1, after a contingency was established in baseline between the observing response and conditioned reinforcement, it was removed and the schedule-correlated stimuli were presented independently of responding according to a variable-time schedule. The variable-time schedule was constructed such that the rate of stimulus presentations was yoked from baseline. The removal of the observing contingency reliably reduced rates of observing. In Experiment 2, resetting delays to conditioned reinforcement were imposed between observing responses and the schedule-correlated stimuli they produced. Delay values of 0, 0.5, 1, 5, and 10 s were examined. Rates of observing varied inversely as a function of delay value. In Experiment 3, signaled and unsignaled resetting delays between observing responses and schedule-correlated stimuli were compared. Baseline rates of observing were decreased less by signaled delays than by unsignaled delays. Disruptions in response-conditioned-reinforcer relations produce similar behavioral effects to those found with primary reinforcement.     

Lesions of the intermediate medial hyperstriatum ventrale impair sickness-conditioned learning in day-old chicks.

Lesion studies show that the intermediate medial hyperstriatum ventrale (IMHV), a forebrain visual association area in chicks, is involved in learning and memory for one-trial passive avoidance and imprinting. We examined the effects of IMHV lesions in a one-trial, nongustatory, sickness-conditioned learning task. This task is similar to passive avoidance and imprinting because all three tasks require the chick to remember visual cues in order to respond correctly. However, sickness-conditioned learning differs from imprinting and passive avoidance because it uses sickness as the aversive stimulus and there is a longer conditioned stimulus-unconditioned stimulus interval (30-min delay compared to seconds). Bilateral IMHV lesions given 24 h before training impaired the ability of the chicks to avoid a bead associated with sickness produced by lithium chloride injection, as did pretraining unilateral left or right IMHV lesions. Post-training IMHV lesions given 1 h after training did not impair avoidance of the test bead in the sickness-conditioned learning task. However, lesioned chicks showed generalized avoidance of all test beads. The pretraining lesion results are similar to those found in imprinting and passive avoidance learning; however, the effects of unilateral IMHV lesions differed. Post-training lesion effects are similar to those found in passive avoidance learning. We propose that both left and right IMHV are necessary for sickness-conditioned learning and that post-training IMHV lesions impair the ability of the chick to learn or remember the association between the color of the bead and the aversive consequences of LiCl injection.     

Responding during reinforcement delay in a self-control paradigm.

Eight pigeons chose between a small, immediate reinforcer and a large, increasingly delayed reinforcer. Responding during the large-reinforcer delays was examined. During large-reinforcer delays, pecks on one key produced the small, immediate reinforcer; pecks on the other key had no effect. Thus, a pigeon could reverse its initial choice of the large, delayed reinforcer, or it could maintain its original choice. Pigeons that made a relatively high number of initial large-reinforcer choices tended to maintain these choices, and those pigeons that actually received a relatively high number of large reinforcers, tended to respond more frequently on the ineffective key during the delay periods. The findings suggest that some previous studies of self-control training in pigeons may have resulted in increased self-control partially due to a lack of opportunity for the pigeons to change their choices.     

Unsignalled delay of reinforcement in variable-interval schedules

Three pigeons responded on several tandem variable-interval fixed-time schedules in which the value of the fixed-time component was varied to assess the effects of different unsignalled delays of reinforcement. Actual (obtained) delays between the last key peck in an interval and reinforcement were consistently shorter than the nominal (programmed) delay. When nominal delays were relatively short, response rates were higher during the delay condition than during the corresponding nondelay condition. At longer nominal delay intervals, response rates decreased monotonically with increasing delays. The results were consistent with those obtained from delay-of-reinforcement procedures that impose either a stimulus change (signal) or a no-response requirement during the delay interval.     

Aversive learning overcomes appetitive innate responding in honeybees

Despite their miniature brain, honeybees have emerged as a powerful model for the study of learning and memory. Yet, they also exhibit innate responses to biologically relevant social signals such as pheromones. Here, we asked whether the bees’ developed learning capabilities allow them to overcome hardwired appetitive responses. Can they learn that attractant pheromones, that are not normally associated with a noxious stimulation in nature, predict the punishment of an electric shock? Immobilized honeybees were trained to discriminate two odorants, one that was paired with a shock and another that had no consequences. We measured whether they learned to produce aversive sting extension responses to the punished but not the non-punished odorant. One odorant was a neutral odor without innate value while the other was either an attractive pheromone (geraniol or citral) or an attractive floral odorant (phenylacetaldehyde). In all cases, bees developed a conditioned aversive response to the punished odorant, be it pheromone or not, and efficiently retrieved this information 1 h later. No learning asymmetries between odors were found. Thus, associative aversive learning in bees is strong enough to override preprogrammed responding, thus reflecting an impressive behavioral flexibility.     

Pigeons' memory for number of events: Effects of intertrial interval and delay interval illumination

In Experiment 1, pigeons were trained at a 0-s baseline delay to discriminate sequences of light flashes (illumination of the feeder) that varied in number but not time (2f/4s and 8f/4s). During training, the intertrial interval was illuminated by the houselight for Group Light, but it was dark for Group Dark. Testing conducted with dark delay intervals produced a strong choose-small bias in both groups. All birds then received baseline training with a 5-s dark delay and were subsequently tested at shorter and longer dark delays. A choose-small bias was again observed at delays longer than the training delay, while a choose-large bias occurred at delays shorter than the training delay. Differentiating the ambient chamber illumination during the intertrial interval and the delay interval did not attenuate choose-small or choose-large effects. In Experiment 2, all birds received baseline training with a 5-s illuminated delay and were subsequently tested at shorter and longer illuminated delays. A choose-large bias was observed at delays longer than the training delay, while a choose-small bias occurred at delays shorter than the training delay. In Experiment 3, on intermittent test trials, when the duration of the second flash on small-sample trials was equal to the total flash duration on large-sample trials (i.e., 1600 ms), accuracy fell to approximately chance. These results suggest that pigeons discriminated the sequences of light flashes that varied in number but not in total duration of the sequence by relying on other temporal properties of the sequence rather than by using an event switch to count flashes.     

Is depression the past tense of anxiety? An empirical study of the temporal distinction

It has long been recognised that depression and anxiety share a common core of negative affect, but research on similarities and differences between these two emotions is growing. The focus of the current study was on whether the timing of a triggering event can determine whether the dominant emotional reaction is depression or anxiety. It was hypothesised that aversive events in the past would elicit more depression than anxiety, whereas the same aversive events in the future would elicit more anxiety than depression. We created temporally varied versions of vignettes describing aversive events occurring at either time, and asked participants to rate the extent to which the events would elicit feelings of depression or anxiety. Results indicated that adverse past events elicited much higher ratings of anticipated depression and adverse future events elicited much higher ratings of anticipated anxiety. Implications for understanding these two emotions and depressive and anxiety disorders are discussed.     

Visual prediction: psychophysics and neurophysiology of compensation for time delays

A necessary consequence of the nature of neural transmission systems is that as change in the physical state of a time-varying event takes place, delays produce error between the instantaneous registered state and the external state. Another source of delay is the transmission of internal motor commands to muscles and the inertia of the musculoskeletal system. How does the central nervous system compensate for these pervasive delays? Although it has been argued that delay compensation occurs late in the motor planning stages, even the earliest visual processes, such as phototransduction, contribute significantly to delays. I argue that compensation is not an exclusive property of the motor system, but rather, is a pervasive feature of the central nervous system (CNS) organization. Although the motor planning system may contain a highly flexible compensation mechanism, accounting not just for delays but also variability in delays (e.g., those resulting from variations in luminance contrast, internal body temperature, muscle fatigue, etc.), visual mechanisms also contribute to compensation. Previous suggestions of this notion of "visual prediction" led to a lively debate producing re-examination of previous arguments, new analyses, and review of the experiments presented here. Understanding visual prediction will inform our theories of sensory processes and visual perception, and will impact our notion of visual awareness.     

The spatiotemporal distinctiveness of direct causation

The launching effect, in which people judge one object to have caused another to immediately move after contact, is often described as the prototype of direct causation. The special status of this interaction may be due to its psychophysical distinctiveness, and this property may be the origin of the formation of causality as a conceptual category. This hypothesis was tested by having participants judge the relative similarity of pairs of events that either had no spatial gap or delay (direct launching) or had small gaps and/or short delays. Direct launching was much easier to discriminate from launches involving small gaps or delays. In a follow-up experiment, participants made similar judgments for a noncausal event.