Connectionist models of conditioning: A tutorial

Models containing networks of neuron-like units have become increasingly prominent in the study of both cognitive psychology and artificial intelligence. This article describes the basic features of connectionist models and provides an illustrative application to compound-stimulus effects in respondent conditioning. Connectionist models designed specifically for operant conditioning are not yet widely available, but some current learning algorithms for machine learning indicate that such models are feasible. Conversely, designers for machine learning appear to have recognized the value of behavioral principles in producing adaptive behavior in their creations.     

The role of observing and attention in establishing stimulus control.

Early theorists (Skinner, Spence) interpreted discrimination learning in terms of the strengthening of the response to one stimulus and its weakening to the other. But this analysis does not account for the increasing independence of the two performances as training continues or for increases in control by dimensions of a stimulus other than the one used in training. Correlation of stimuli with different densities of reinforcement produces an increase in the behavior necessary to observe them, and greater observing of and attending to the relevant stimuli may account for the increase in control by these stimuli. The observing analysis also encompasses errorless training, and the selective nature of observing explains the feature-positive effect and the relatively shallow gradients of generalization generated by negative discriminative stimuli. The effectiveness of the observing analysis in handling these special cases adds to the converging lines of evidence supporting its integrative power and thus its validity.     

Sequential effects in dimensional contrast

Two experiments examined pigeons' response rates during short trials signaled by stimuli closely spaced along a wavelength continuum. In Experiment 1 separate halves of the continuum were correlated with different reinforcement schedules. In Experiment 2, the middle stimulus was accompanied by a lower probability of reinforcement than were the remaining wavelengths. Both procedures resulted in dimensional contrast “shoulders,” seen as relatively enhanced or depressed response rates in the presence of stimuli between the extreme of the continuum and the border separating the positive and negative stimuli. Sequential analyses addressed possible contributions of the following interactions: (a) local contrast, seen when rate during a given schedule depends on the schedule in the just-preceding trial; (b) modification of local contrast by the similarity of the signaling stimuli (P. Blough, 1983); and (c) schedule-independent rate contrast, seen when rate in a given trial depends on the rate controlled by the stimulus that accompanied the just-preceding trial (Malone & Rowe, 1981). Dimensional contrast functions were similar when isolated according to the schedule, to the similarity of the signaling stimulus, and to the response rate of the just-preceding trial. The interactions noted above do not appear to make important contributions to this effect.     

Stimulus class membership established via stimulus-reinforcer relations

In an arbitrary matching-to-sample procedure, two mentally retarded subjects learned conditional discriminations with two sets of stimuli. Each set included a spoken name (N1 or N2), an object (O1 or O2), and a printed symbol (S1 or S2). One subject selected conditionally (a) O1 upon N1, and O2 upon N2, and (b) S1 upon O1, and S2 upon O2. The other subject selected conditionally (a) S1 upon N1, and S2 upon N2, and (b) O1 upon S1, and O2 upon S2. For both subjects, selections of O1 and S1 produced one type of food, F1; selections of O2 and S2 produced a different type of food, F2. Both subjects also learned identity-matching performances, selecting O1, O2, S1, S2, F1, and F2 conditionally upon those stimuli as samples; F1 followed selections of O1, S1, and F1; F2 followed selections of O2, S2, and F2. Matching performances consistent with stimulus class formation involving the names, objects, symbols, and foods were demonstrated on probe trials, even though these performances had not been taught explicitly. Next, new objects, X1 and X2, were presented on identity-matching trials, producing F1 and F2, respectively. Without further training, X1 was selected conditionally upon N1, S1, and O1, and X2 was selected upon N2, S2, and O2. When the contingencies were changed so that selections of X1 and X2 were now followed by F2 and F1, respectively, X2 was selected conditionally upon N1, S1, and O1, and X1 was selected upon N2, S2, and O2. Class membership of X1 and X2 had apparently changed. This study provides evidence that reinforcers may become members of stimulus classes, and that new stimuli may become class members through relations with reinforcers.     

Control of responding during stimuli that precede transitions in reinforcement frequency

Pigeons' responses were reinforced on a variant of a mixed variable-interval extinction schedule of reinforcement in which the transition to the higher reinforcement rate was signaled by a trace stimulus projected on the response key prior to the onset of the component correlated with food delivery. In the first of two experiments, the duration of the trace stimulus preceding the component correlated with food delivery was varied from 1.5 to 50.0 s and in the second experiment, the reinforcement frequency in the same component was varied from 10 to 60 reinforcers per hour. Pigeons pecked at the trace stimulus preceding the onset of the component correlated with food delivery even though responding was not reinforced in its presence and only one of the changes in reinforcement rate (i.e., from extinction to reinforcement) was signaled. The rate of pecking during the trace stimulus was a function of its duration but not of the reinforcement frequency in the following component. Higher rates generally occurred at the shorter trace-stimulus durations. Component responding following the offset of the trace stimulus was under discriminative control of the trace stimulus whether or not responding occurred in the presence of the trace stimulus.     

Conditional relations by monkeys: Reflexivity, symmetry, and transitivity

Two cynomolgous macaques categorized six colors into two groups of three after conditional discrimination training (zero-delay symbolic match-to-sample). The procedures resulted in the establishment of relations among the elements of each set-relations that were not specifically trained and that can be characterized by the properties of reflexivity, symmetry, and transitivity. Each set of colors was related to a characteristic pattern of responding: One response pattern involved temporal duration (press and hold the response keys); the second response pattern entailed repeated pressing and releasing of the response keys (fixed ratio 8). Six combinations of two colors were trained, three combinations from each set. After discriminative performance stabilized for each monkey, they were tested with 10 additional color combinations, all of which differed from the training combinations. The conditional relations established between test combinations can be characterized as stimulus equivalence. The training procedures were analogous to the procedure of using category names, and have implications for understanding the function of language in the formation of equivalence classes.     

Observational learning of two visual discriminations by pigeons: a within-subjects design.

Pigeon's observational learning of successive visual discrimination was studied using within-subject comparisons of data from three experimental conditions. Two pairs of discriminative stimuli were used; each bird was exposed to two of the three experimental conditions, with different pairs of stimuli used in a given bird's two conditions. In one condition, observers were exposed to visual discriminative stimuli only. In a second condition, subjects were exposed to a randomly alternating sequence of two stimuli where the one that would subsequently be used as S+ was paired with the operation of the grain magazine. In a third experimental condition, subjects were exposed to the performance of a conspecific in the operant discrimination procedure. After exposures to conspecific performances, there was facilitation of discriminative learning, relative to that which followed exposures to stimulus and reinforcement sequences or exposures to stimulus sequences alone. Exposure to stimulus and food-delivery sequences enhanced performance relative to exposure to stimulus sequences alone. The differential effects of these three types of exposure were not attributable to order effects or to task difficulty; rather, they clearly were due to the type of exposure.     

The effect of negative stimulus presentations on observing-response rates

Theories of observing differ in predicting whether or not a signal for absence of reinforcement (S−) is capable of reinforcing observing responses. Experiments in which S− was first removed from and then restored to the procedure have yielded mixed results. The present experiments suggest that failure to control for the direct effect of presenting S− may have been responsible. Pigeons and operant procedures were used. Experiment 1 showed that presentations of S−, even when not contingent on observing, can raise the rate of an observing response that was reinforced only by presentations of a signal (S+) that accompanied a schedule of food delivery. Experiment 2 showed that this effect resulted from bursts of responding that followed offsets of S−. Experiment 3 showed that, when the presence of S− was held constant, lower rates occurred when S− was dependent on, rather than independent of, observing. These results support theories that characterize S− as incapable of reinforcing observing responses.     

Temporal control by signals of interval duration within variable-interval schedules

Rats' lever pressing produced sucrose reinforcers on a variable-interval schedule where, in different conditions, the duration of a stimulus presented immediately after reinforcement was either correlated or uncorrelated with the duration of the current interreinforcement interval. Under the baseline schedule, in which no stimulus was presented, the minimum interreinforcement interval was 8 s and the mean postreinforcement pause of each subject approximated this value. Response rates increased slowly over the first 10 to 15 s and then remained roughly constant throughout the remainder of the interval. In both the correlated and uncorrelated conditions, the added stimulus resulted in the postreinforcement pauses lengthening to values in excess of the duration of the preceding stimulus. This resulted in a poststimulus pause which was, in most cases, roughly constant irrespective of the duration of the preceding stimulus, or of the reinforcement contingencies prevailing immediately after stimulus offset. Local response-rate patterns in the uncorrelated conditions were similar to those obtained under the baseline schedule in which no stimulus was presented. However, in the correlated condition local response rates increased across the remainder of the interreinforcer interval. Further, the rate of acceleration was inversely related to the duration of the preceding stimulus. These results show that a correlation between stimulus duration and the ensuing time to reinforcement can control behavior—a type of temporal control not previously reported.     

Empirical tests of scientific realism: A quantitative framework

The scientific realism debate in philosophy of science raises some intriguing methodological issues. Scientific realism posits a link between a scientific theory's observational and referential success. This opens the possibility of testing the thesis empirically, by searching for evidence of such a link in the record of theories put forward in the history of science. Many realist philosophers working today propose case study methodology as a way of carrying out such a test. This article argues that a qualitative method such as case study methodology is not adequate for this purpose, for two reasons: to test scientific realism is to pose an effects-of-causes question, and observational and referential success are quantities that theories possess to a greater or lesser degree. The article concludes that an empirical test of scientific realism requires a quantitative method.