The Rescorla-Wagner model: losses in associative strength in compound conditioned stimuli

This series of experiments demonstrated that compound conditioned stimuli (CSs) highly conditioned in isolation lose conditioned response strength when compound-CS-unconditioned stimulus (US) pairings are given. This loss in conditioned responding was a positive function of the number of compound-CS-US pairings and was greater for the more salient stimulus. Finally, if a previously neutral stimulus was compounded with two previously conditioned cues, the neutral stimulus could acquire conditioned inhibitory properties with only compound-CS-US pairings. These results provide support for some aspects of the Rescorla-Wagner model.     

Stimulus generalization: some predictions from a model of Pavlovian conditioning.

Three experiments examined predictions generated by incorporating a common-elements account of stimulus generalization within the Rescorla-Wagner model of conditioning. All experiments employed rats in a conditioned suppression situation. Experiments 1 and 2 found that conditioning of a similar stimulus augmented the excitation controlled by a near-asymptotic target stimulus more than did further conditioning of the target itself. Prior discrimination training between the target and the similar stimulus enlarged this effect, compared with prior discrimination between the target and another dissimilar stimulus. Nonreinforced exposure of the similar stimulus prior to its reinforcement also increased the effect. Experiment 3 examined a related prediction for inhibition. After discrimination training, extinction of the previously reinforced stimulus revealed more inhibition to the previously nonreinforced stimulus when those two stimuli were more similar. These outcomes are consistent with deductions from the present model and encourage further testing of its expansion to the case of stimulus generalization.     

Initial excitatory conditioning with the truly random control procedure in rats: The effects of density of the conditioned stimulus

The present experiments were designed to test whether initial excitatory conditioning to a stimulus was obtained, using a within-subject design of the truly random control (TRC) procedure, and to examine the effects of the density of the conditioned stimulus on it. In Experiment 1, the effect of the three levels of CS density (50%, 25%, 20%) was assessed at six different stages of training, using the conditioned suppression of licking in rats. Although all of the groups given the TRC procedure showed evidence of initial excitatory conditioning, it was more marked in the group with the highest CS density than in the other groups. Experiment 2 confirmed the results of Experiment 1, using a between-subject design. The findings are in line with predictions from the Rescorla-Wagner model, and are discussed with reference to other contemporary theories of associative learning.     

Blocking but not conditioned inhibition results when an added stimulus is reinforced in compound with multiple pretrained stimuli

Conditioned barpress suppression by rats was used to explore the associative status of an initially neutral stimulus that was reinforced in simultaneous compound with two independently pretrained conditioned excitors. In contrast to the Rescorla-Wagner model, which predicts that the target stimulus (X) will be inhibitory following such A+/B+/ABX+ training, the present study found no evidence that X acquired inhibitory associative strength. Rather, the pretrained stimuli merely served to block conditioned excitatory responding to the target stimulus.     

Is the number of trials a primary determinant of conditioned responding?

Acquisition of conditioned responding is thought to be determined by the number of pairings of a conditioned stimulus (CS) and an unconditioned stimulus (US). However, it is possible that acquisition is primarily determined not by the number of trials but rather by quantities that often correlate with the number of trials, such as cumulative intertrial interval (ITI) and the number of sessions. Four experiments examined whether the number of trials has an effect on acquisition of conditioned responding, once cumulative ITI and number of sessions are equated. Results of the experiments with rats and mice favor the hypothesis that over an eightfold range, variation in number of CS-US pairings has little effect. It is suggested that learning curves might more accurately be plotted across cumulative ITI or number of sessions, and not across number of trials. Results pose a challenge to trial-centered accounts of conditioning, as demonstrated by simulations of the Rescorla-Wagner model, a simplified version of Wagner's standard operating procedure model (SOP), and Stout & Miller's sometimes competing retrieval model (SOCR). A time-centered account, rate estimation theory (RET), predicts the main finding but has trouble with other aspects of the learning process more easily accommodated by trial-centered models.     

Contextual conditioning

Eastern and Western interpretations of contextual control of phasic conditional responses (transswitching) are contrasted. The Eastern (Asratyan, 1965) approach emphasizes the role of the tonic conditional stimulus and the (hypothetical) tonic response it evokes. The Western (Lachnit, 1986) approach emphasizes the role of compound conditional stimuli. Although Lachnit showed that transswitching-like results can be obtained without a tonic stimulus, attempts to simulate transswitching experiments using a computer model of the Rescorla-Wagner theory (Kimmel and Lachnit, 1988) have shown that predictions from the theory approximate empirical results in human classical conditioning only when the tonic stimulus is given far greater weight than the phasic stimulus. In other words, only when the Rescorla-Wagner theory is made more like Asratyan’s theory, can the compound conditional stimulus approach account for real empirical transswitching data.     

Making causal judgments from the proportion of confirming instances: the pCI rule

It is proposed that causal judgments about contingency information are derived from the proportion of confirmatory instances (pCI) that are evaluated as confirmatory for the causal candidate. In 6 experiments, pCI values were manipulated independently of objective contingencies assessed by the deltaP rule. Significant effects of the pCI manipulations were found in all cases, but causal judgments did not vary significantly with objective contingencies when pCI was held constant. The experiments used a variety of stimulus presentation procedures and different dependent measures. The power PC theory, a weighted version of the deltaP rule, the Rescorla-Wagner associative learning model (R. A. Rescorla & A. R Wagner, 1972), and the deltaD rule, which is the frequency-based version of the pCI rule, were unable to account for the significant effects of the pCI manipulations. These results are consistent with a general explanatory approach to causal judgment involving the evaluation of evidence and updating of beliefs with regard to causal hypotheses.     

Further Investigation of Responding Elicited by BC and C After A+, AB-, ABC+ Training

Two experiments with pigeons explored conditioned keypeck responding to new visual targets after visual compound discrimination training. In the first experiment, pigeons were trained with an A+, AB-, ABC+, AD-, ADC+ task, in which stimulus A signalled food, stimulus compounds AB and AD signalled no food, and stimulus compounds ABC and ADC signalled food. According to both an elemental model (Rescorla & Wagner, 1972) and a configural model (Pearce, 1987) of Pavlovian conditioning, test compounds BC and DC should elicit less responding than should C alone. However, the birds responded more to BC and DC than to C. In the second experiment, another set of pigeons was trained with an A+, AB-, ABC+, AD-, ADE+ task, in which stimulus A signalled food, stimulus compounds AB and AD signalled no food, and stimulus compounds ABC and ADE signalled food. Contrary to the prediction of the Rescorla-Wagner model, keypeck responding was not less on BC and DE trials than on C and E trials in testing. However, B and D attenuated responding to E and C, respectively, when presented in compounds BE and DC. The Pearce model was able to accommodate these results.     

From conditioning to category learning: an adaptive network model

We used adaptive network theory to extend the Rescorla-Wagner (1972) least mean squares (LMS) model of associative learning to phenomena of human learning and judgment. In three experiments subjects learned to categorize hypothetical patients with particular symptom patterns as having certain diseases. When one disease is far more likely than another, the model predicts that subjects will substantially overestimate the diagnosticity of the more valid symptom for the rare disease. The results of Experiments 1 and 2 provide clear support for this prediction in contradistinction to predictions from probability matching, exemplar retrieval, or simple prototype learning models. Experiment 3 contrasted the adaptive network model with one predicting pattern-probability matching when patients always had four symptoms (chosen from four opponent pairs) rather than the presence or absence of each of four symptoms, as in Experiment 1. The results again support the Rescorla-Wagner LMS learning rule as embedded within an adaptive network model.     

Asymptotic judgment of cause in a relative validity paradigm

We report three experiments in which we tested asymptotic and dynamic predictions of the Rescorla-Wagner (R-W) model and the asymptotic predictions of Cheng's probabilistic contrast model (PCM) concerning judgments of causality when there are two possible causal candidates. We used a paradigm in which the presence of a causal candidate that is highly correlated with an effect influences judgments of a second, moderately correlated or uncorrelated cause. In Experiment 1, which involved a moderate outcome density, judgments of a moderately positive cause were attenuated when it was paired with either a perfect positive or perfect negative cause. This attenuation was robust over a large set of trials but was greater when the strong predictor was positive. In Experiment 2, in which there was a low overall density of outcomes, judgments of a moderately correlated positive cause were elevated when this cause was paired with a perfect negative causal candidate. This elevation was also quite robust over a large set of trials. In Experiment 3, estimates of the strength of a causal candidate that was uncorrelated with the outcome were reduced when it was paired with a perfect cause. The predictions of three theoretical models of causal judgments are considered. Both the R-W model and Cheng's PCM accounted for some but not all aspects of the data. Pearce's model of stimulus generalization accounts for a greater proportion of the data.     

Overprediction and blocking in the McCollough aftereffect

The Rescorla-Wagner theory (Rescorla & Wagner, 1972) of associative learning offers specific predictions about the associative strength of CS-US pairs when two or more CSs are conditioned to the same US separately and subsequently paired in a compound with the same US. The magnitude of orientation-contingent color aftereffects (AEs) was used as an index of associative strength in this study. The results of experiments using an "overprediction" (Rescorla, 1970) and a "blocking" (Kamin, 1969) paradigm conformed to the predictions of the Rescorla-Wagner theory. In Experiment 1, AEs were established simultaneously for horizontal-vertical and diagonal patterns. When observers subsequently viewed compound induction patterns, AE magnitude was found to be significantly decreased, relative to a condition in which observers did not view such an induction pattern. In Experiment 2, AE magnitude for a given test pattern following inspection of compound induction stimuli was significantly reduced by inspection of the other component prior to viewing the compound induction stimuli. The applicability of associative learning and feature-adaptation models of the McCollough effect is discussed.     

Control of behavior by an establishing stimulus

Seventeen pigeons were exposed to a three-key discrete-trial procedure in which a peck on the lit center key produced food if, and only if, the left keylight was lit. The center key was illuminated by a peck on the lit right key. Of interest was whether subjects pecked the right key before or after the response-independent onset of the left keylight. Pecks on the right key after left-keylight onset suggest control of behavior by the left keylight—an establishing stimulus. In three experiments, the strength of center-keylight onset as conditioned reinforcer for a response on the right key was manipulated by altering the size of the reduction in time to food delivery correlated with its onset. Control of pigeons' key pecks by onset of the left keylight occurred on more trials per session when the center keylight was a relatively weak conditioned reinforcer and on fewer trials per session when the center keylight was a relatively strong condtioned reinforcer. Differences across conditions in the degree of control by onset of the establishing stimulus were greatest when changes in conditioned reinforcer strength occurred relatively frequently and were signaled. The results provide evidence of the function of an establishing stimulus.     

Parallel Neural Systems for Classical Conditioning: Support From Computational Modeling

Classical conditioning has been explained by two main types of theories that postulate different learning mechanisms. Rescorla and Wagner (1972) put forth a theory in which conditioning is based on the ability of the US to drive learning through error correction. Alternatively, Mackintosh (1973) put forth a theory in which the ability of the CS to be associated with the unconditioned stimulus is modulated. We have proposed a reconciliation of these two mechanisms as working in parallel within different neural systems: a cerebellar system for US modulation and a hippocampal system for CS modulation. We developed a computational model of cerebellar function in eyeblink conditioning based on the error correction mechanism of the Rescorla-Wagner rule in which learning-related activity from the cerebellum inhibits the inferior olive, which is the US input pathway to the cerebellum (Gluck et al., 1994). We developed a computational model of the hippocampal region that forms altered representations of conditioned stimuli based on their behavioral outcomes (Gluck & Myers, 1993; Myers et al., 1995). Overall, computational modeling and empirical findings support the idea that, at least in the case of eyeblink conditioning, there may be two different neural systems: the cerebellum which mediates US-based error correction and hippocampus which alters representations of CSs.     

Parallel neural systems for classical conditioning: Support from computational modeling

Classical conditioning has been explained by two main types of theories that postulate different learning mechanisms. Rescorla and Wagner (1972) put forth a theory in which conditioning is based on the ability of the US to drive learning through error correction. Alternatively, Mackintosh (1973) put forth a theory in which the ability of the CS to be associated with the unconditioned stimulus is modulated. We have proposed a reconciliation of these two mechanisms as working in parallel within different neural systems: a cerebellar system for US modulation and a hippocampal system for CS modulation. We developed a computational model of cerebellar function in eyeblink conditioning based on the error correction mechanism of the Rescorla-Wagner rule in which learningrelated activity from the cerebellum inhibits the inferior olive, which is the US input pathway to the cerebellum (Gluck et al., 1994). We developed a computational model of the hippocampal region that forms altered representations of conditioned stimuli based on their behavioral outcomes (Gluck & Myers, 1993; Myers et al., 1995). Overall, computational modeling and empirical findings support the idea that, at least in the case of eyeblink conditioning, there may be two different neural systems: the cerebellum which mediates US-based error correction and hippocampus which alters representations of CSs.     

Asymmetrical effects of Pavlovian excitatory and inhibitory aversive transfer on Pavlovian appetitive responding and acquisition

Predictions of a theory of Pavlovian motivational transfer, which incorporates principles of both the theory of reciprocal inhibition and the Rescorla-Wagner model, were tested in several Pavlovian aversive to Pavlovian appetitive transfer tasks. As predicted, the presence of a signal for an aversive event, conditioned stimulus (AV CS+), reliably suppressed performance of appetitive conditioned responses (CRs) whether imposed during acquisition or on independently established responding. Acquisition of appetitive responding to a novel CS reinforced in compound with an AV CS+, however, was enhanced (“superconditioning”). This observation suggests that the effects of a discrepancy between expectation and actual outcome on a conditioning trial are influenced by the affective value of both the expectation and the reinforcer. These transfer effects were not symmetrical for an inhibitory aversive stimulus (AV CS?). An AV CS? did not enhance appetitive responding compared to a random control condition, nor did the AV CS? reduce (i.e., block) appetitive conditioning to a novel CS when appetitive reinforcement occurred in the presence of the AV CS?. Comparison of the two shock-exposed conditions with a naive control condition suggests that previous results that were apparently consistent with inhibitory aversive enhancement and blocking of appetitive conditioning may have been due to aversive context conditioning.     

Visual stimulus intensity and location probability: Interactive effects on choice reaction time

Three prevalent models predict different stimulus intensity effects on RT. These are: the serial-stage model (additivity), the variable criterion model (larger intensity effects with slower responding), and the temporal overlap model (smaller intensity effects with slower responding). The predictions were tested in a dual-response situation including oculomotor and manual responses (RT). Other variables were stimulus location probability (expectancy), foreperiod (alertness), and stimulus intensity (encoding. In Experiment I, an Intensity x Probability interaction was found such that the intensity-effect was smaller at low stimulus probability. Three further experiments were performed in order to specify some of the conditions relevant for this phenomenon. The results are consistent with the temporal-overlap interpretation. It was suggested that the obtained interaction results from a processing delay due to increased demands for cognitive and response processing (Stanovich & Pachella, 1977), or to the attention switch to an unexpected S-R event.     

注意力资源限制与双任务的相互干扰机制

采用心理不应期研究范式, 两个反应时实验检测了注意力资源分配的特征以及双任务的相互干扰机制。每次实验中, 要求被试快速、相继对高低音辨别任务(T1)和Stroop任务(T2)作出选择性反应, T1和T2间采用6种不同的时间间隔(SOA), 以系统考察不同SOA条件下两个任务的反应时走势。结果发现：(1) 在重叠的双任务情境中, T1的中枢加工导致在T2上出现显著的PRP效应, T2的中枢反应选择对T1的反应选择和反应执行加工同样产生显著的影响。SOA以及T2的难度与复杂度实质性地影响了T1的反应选择和反应执行加工。(2) 当两个任务同时需要进行中枢反应选择加工时, 一个任务占用更多的注意资源将导致另一任务获得较少的注意资源, 注意资源量的多寡直接决定了该任务的加工效率。(3) 两个任务的加工相互影响、相互制约, 这种制约机制不仅仅存在于中枢反应选择阶段, 在反应执行阶段仍然存在。     

Psychophysiologic chronometry: detection of facilitating and inhibiting processes in information processing using electroencephalographic and electromyographic data]

The objective of the present research was to tally the potential usefulness of psychophysiological measures in the systematic analysis of human information processing. Different context conditions were established in two cued choice-reaction time experiments. The context could either facilitate or inhibit the processing of an imperative stimulus. Event-related brain potentials (EEG-ERP) and event-related electromyograms (EMG-ERP) were recorded and from these biosignals latencies and duration times for different aspects of stimulus evaluation and movement execution were derived. The EEG-ERP provided measures for the time of primary stimulus categorization (N2), the time necessary to test actual stimulus information against stimulus expectancies (P3b), and the time necessary to check the decision about the stimulus-response mapping (pSW). From the EMG-ERP, processing times were derived which are related to the initiation of a motor program and to distinct sections of the movement trajectory. Results show that these latency measures react differently to different context manipulations. EEG-ERP measures, for example, reflected differences during stimulus evaluation which were not detectable in response- or EMG-derived movement times. The parallel analysis of EEG-ERPs and EMG-ERPs allowed delineation of processing modules which are responsible for particular context-dependent facilitation and inhibition effects. Moreover, the data revealed substantial overlap of evaluation times and movement times. This finding seems to be more compatible with a parallel processing model than with a strictly sequential model.     

Observing response acquisition: preference for unpredictable appetitive rewards obtained under conditions predicted by DMOD

In five E-maze experiments, rats were given a choice between receiving reward and nonreward in a situation where stimuli were correlated with reward outcome (predictable situation) versus one where the stimuli were uncorrelated with reward outcome (unpredictable situation). Preference for the unpredictable situation occurred under the following conditions: (a) small (one 37-mg pellet), immediate rewards; (b) small, delayed (15 s) rewards, if the cues correlated with reward outcome were absent during the delay interval; (c) large (15 pellets), immediate rewards if a difficult discrimination was required; and (d) if the stimulus predicting nonreward was present at the choice point. Preference for the predictable situation was strongest if reinforcement was delayed and large or the stimulus predicting reward was present at the choice point. A weaker preference for the predictable situation occurred if reinforcement was immediate and large and a simple discrimination was required or if reinforcement was large and delayed and the cues that correlated with reward outcome were absent during the delay interval. The results support the predictions of DMOD (Daly modification of the Rescorla-Wagner model), a mathematical model of appetitive learning (Daly & Daly, 1982).     

Parallel processing and the psychological refractory period

The serial processor theory proposed by Welford is the traditional explanation of the delay in responding to the second stimulus presented in psychological refractory period paradigm experiments. This paper reviews results which do not accord either with that theory or with later modifications of it by Smith and Tolkmitt. To explain these results a model is proposed which contains three propositions: (a) parallel processing of concurrent stimuli, (b) fixed total processing capacity, (c) capacity allocation either dependent on the relative difficulty of the stimuli or set by the subject according to experimental instructions. The predictions of this model for the effects of stimulus difficulty and interstimulus interval on reaction time and interresponse interval are computed. These predictions are compared with those of other parallel processing models of the psychological refractory period which have been proposed by Herman and Kantowitz, by Kahneman and by Keele.