REFLECTIONS ON BEHAVIOR ANALYSIS AND EVOLUTIONARY BIOLOGY: A SELECTIVE REVIEW OF EVOLUTION SINCE DARWIN—THE FIRST 150 YEARS. EDITED BY M. A. BELL,D. J. FUTUYAMA,W. F. EANES, & J. S. LEVINTON

This review focuses on parallels between the selectionist sciences of evolutionary biology and behavior analysis. In selectionism, complex phenomena are interpreted as the cumulative products of relatively simple processes acting over time—natural selection in evolutionary biology and reinforcement in behavior analysis. Because evolutionary biology is the more mature science, an examination of the factors that led to the triumph of natural selection provides clues whereby reinforcement may achieve a similar fate in the science of behavior.     

Behavior analysis and neuroscience: Complementary disciplines

Behavior analysis and neuroscience are disciplines in their own right but are united in that both are subfields of a common overarching field—biology. What most fundamentally unites these disciplines is a shared commitment to selectionism, the Darwinian mode of explanation. In selectionism, the order and complexity observed in nature are seen as the cumulative products of selection processes acting over time on a population of variants—favoring some and disfavoring others—with the affected variants contributing to the population on which future selections operate. In the case of behavior analysis, the central selection process is selection by reinforcement; in neuroscience it is natural selection. The two selection processes are inter‐related in that selection by reinforcement is itself the product of natural selection. The present paper illustrates the complementary nature of behavior analysis and neuroscience through considering their joint contributions to three central problem areas: reinforcement—including conditioned reinforcement, stimulus control—including equivalence classes, and memory—including reminding and remembering.     

Behavior analysis and revaluation

Revaluation refers to phenomena in which the strength of an operant is altered by reinforcer-related manipulations that take place outside the conditioning situation in which the operant was selected. As an example, if lever pressing is acquired using food as a reinforcer and food is later paired with an aversive stimulus, the frequency of lever pressing decreases when subsequently tested. Associationist psychology infers from such findings that conditioning produces a response-outcome (i.e., reinforcer) association and that the operant decreased in strength because pairing the reinforcer with the aversive stimulus changed the value of the outcome. Here, we present an approach to the interpretation of these and related findings that employs neural network simulations grounded in the experimental analysis of behavior and neuroscience. In so doing, we address some general issues regarding the relations among behavior analysis, neuroscience, and associationism.     

Directional harmonic theory: a computational Gestalt model to account for illusory contour and vertex formation

Visual illusions and perceptual grouping phenomena offer an invaluable tool for probing the computational mechanism of low-level visual processing. Some illusions, like the Kanizsa figure, reveal illusory contours that form edges collinear with the inducing stimulus. This kind of illusory contour has been modeled by neural network models by way of cells equipped with elongated spatial receptive fields designed to detect and complete the collinear alignment. There are, however, other illusory groupings which are not so easy to account for in neural network terms. The Ehrenstein illusion exhibits an illusory contour that forms a contour orthogonal to the stimulus instead of collinear with it. Other perceptual grouping effects reveal illusory contours that exhibit a sharp corner or vertex, and still others take the form of vertices defined by the intersection of three, four, or more illusory contours that meet at a point. A direct extension of the collinear completion models to account for these phenomena tends towards a combinatorial explosion, because it would suggest cells with specialized receptive fields configured to perform each of those completion types, each of which would have to be replicated at every location and every orientation across the visual field. These phenomena therefore challenge the adequacy of the neural network approach to account for these diverse perceptual phenomena. I have proposed elsewhere an alternative paradigm of neurocomputation in the harmonic resonance theory (Lehar 1999, see website), whereby pattern recognition and completion are performed by spatial standing waves across the neural substrate. The standing waves perform a computational function analogous to that of the spatial receptive fields of the neural network approach, except that, unlike that paradigm, a single resonance mechanism performs a function equivalent to a whole array of spatial receptive fields of different spatial configurations and of different orientations, and thereby avoids the combinatorial explosion inherent in the older paradigm. The present paper presents the directional harmonic model, a more specific development of the harmonic resonance theory, designed to account for specific perceptual grouping phenomena. Computer simulations of the directional harmonic model show that it can account for collinear contours as observed in the Kanizsa figure, orthogonal contours as seen in the Ehrenstein illusion, and a number of illusory vertex percepts composed of two, three, or more illusory contours that meet in a variety of configurations.     

Behavioral momentum theory: equations and applications

Behavioral momentum theory provides a quantitative account of how reinforcers experienced within a discriminative stimulus context govern the persistence of behavior that occurs in that context. The theory suggests that all reinforcers obtained in the presence of a discriminative stimulus increase resistance to change, regardless of whether those reinforcers are contingent on the target behavior, are noncontingent, or are even contingent on an alternative behavior. In this paper, we describe the equations that constitute the theory and address their application to issues of particular importance in applied settings. The theory provides a framework within which to consider the effects of interventions such as extinction, noncontingent reinforcement, differential reinforcement of alternative behavior, and other phenomena (e.g., resurgence). Finally, the theory predicts some counterintuitive and potentially counterproductive effects of alternative reinforcement, and can serve as an integrative guide for intervention when its terms are identified with the relevant conditions of applied settings.     

Identification of biological neurons using adaptive observers

This paper is to investigate the use of adaptive observers for the modeling of biological neurons and networks. Assuming that a neuron can be modeled as a continuous-time nonlinear system, it is possible to determine its unknown parameters using adaptive observer, based on the concept of adaptive synchronization. The same technique can be extended for the identification of an entire biological neural network. Some conventional observer designs are studied in this paper and satisfactory results are obtained, yet with some restrictions. To further extend the applicability of adaptive observers for the modeling process, a new design is suggested. It is based on a combination of linear feedback control approach and the dynamical minimization algorithm. The effectiveness of the designed adaptive observer is confirmed with simulations.     

Correlations Between Input and Output Units in Neural Networks

Correlation analyses of recent back-propagation neural networks show that network results are due to imbalances in stimulus input. Conclusions concerning the effects of receptive field size, hemispheric specialization, and other issues of relevance to psychology cannot therefore be drawn until the dominating effects of low-level correlations are removed. Statistical techniques for evaluating the stimulus materials for neural networks are introduced.     

Brain mechanisms associated with background monitoring of the environment for potentially significant sensory events

Background monitoring is a necessary prerequisite to detect unexpected changes in the environment, while being involved in a primary task. Here, we used fMRI to investigate the neural mechanisms that underlie adaptive goal-directed behavior in a cued task switching paradigm during real response conflict or, more generally, when expectations on the repetitive features of the environment were violated. Unexpected changes in sensory stimulus attributes in the currently unattended stimulus dimension thereby led to activations in a bilateral network comprising inferior lateral frontal, intraparietal, and posterior medial frontal brain regions, independent of whether these attributes elicited a factual response conflict or not. This fronto-parietal network may thus play an important role in adaptive responding to potentially significant events outside the current focus of attention.     

Signaled alternative reinforcement and the persistence of operant behavior

Differential reinforcement of alternative behavior (DRA) is a treatment designed to eliminate problem behavior by reinforcing an alternative behavior at a higher rate. Availability of alternative reinforcement may be signaled, as with Functional Communication Training, or unsignaled. Whether or not alternative reinforcement is signaled could influence both the rate and persistence of problem behavior. The present study investigated whether signaling the availability of alternative reinforcement affects the rate and persistence of a concurrently available target response with pigeons. Three components of a multiple concurrent schedule arranged equal reinforcement rates for target responding. Two of the components also arranged equal reinforcement rates for an alternative response. In one DRA component, a discrete stimulus signaled the availability of response‐contingent alternative reinforcement by changing the keylight color upon reinforcement availability. In the other DRA component, availability of alternative reinforcement was not signaled. Target responding was most persistent in the unsignaled DRA component when disrupted by satiation, free food presented between components, and extinction, relative to the signaled DRA and control components. These findings suggest the discrete stimulus functionally separated the availability of alternative reinforcement from the discriminative stimuli governing target responding. These findings provide a novel avenue to explore in translational research assessing whether signaling the availability of alternative reinforcement with DRA treatments reduces the persistence of problem behavior.     

Reinforcement contingencies and social reinforcement: some reciprocal relations between basic and applied research.

Reinforcement contingencies and social reinforcement are ubiquitous phenomena in applied behavior analysis. This discussion paper is divided into two sections. In the first section, reinforcement contingencies are discussed in terms of the necessary and sufficient conditions for reinforcement effects. Response‐stimulus dependencies, conditional probabilities, and contiguity are discussed as possible mechanisms of, and arrangements for, reinforcement effects. In the second section, social reinforcement is discussed in terms of its functional subtypes and reinforcement context effects. Two underlying themes run throughout the discussion: (a) Applied research would benefit from a greater understanding of existing basic research, and (b) basic research could be designed to specifically address some of the issues about reinforcement that are central to effective application.     

Designing interventions that include delayed reinforcement: implications of recent laboratory research

The search for robust and durable interventions in everyday situations typically involves the use of delayed reinforcers, sometimes delivered well after a target behavior occurs. Integrating the findings from laboratory research on delayed reinforcement can contribute to the design and analysis of those applied interventions. As illustrations, we examine articles from the Journal of the Experimental Analysis of Behavior that analyzed delayed reinforcement with respect to response allocation (A. M. Williams & Lattal, 1999), stimulus chaining (B. A. Williams, 1999), and self-control (Jackson & Hackenberg, 1996). These studies help to clarify the conditions under which delayed reinforcement (a) exercises control of behavior, (b) entails conditioned reinforcement, and (c) displaces the effects of immediate reinforcement. The research has applied implications, including the development of positive social behavior and teaching people to make adaptive choices. DESCRIPTORS: delayed reinforcement, response allocation, stimulus chains, self-control, integration of basic and applied research     

REVIEW OF THE BOOK SNIFFY THE VIRTUAL RAT PRO VERSION 2.0

Sniffy the Virtual Rat Pro Version 2.0 is the most recent update to the Sniffy the Rat program modules. It has an extensive set of simulations related to Pavlovian and operant conditioning. Many of the standard conditioning paradigms and phenomena are contained within this program. Pavlovian effects such as blocking, overshadowing, and stimulus competition are demonstrated in the data output. Operant manipulations allow shaping the behavior of the virtual rat, observing cumulative records of interval and ratio schedules, as well as conducting operant discrimination procedures. In both Pavlovian and operant paradigms, one can generate histograms that predict the degree to which various stimuli control the virtual rat's behavior. On‐screen presentations may keep most users interested, but working with the generated data files sometimes can prove difficult. Overall, this program has a strong potential for facilitating the instruction in undergraduate conditioning courses, serving as an addendum to traditional undergraduate conditioning laboratories and as a supplement to the use of live animals.     

Conjoint control of performance in conditional discriminations by successive and simultaneous stimuli.

In a conditional discrimination, reinforcement of pigeons' responses to pairs of simultaneously presented wavelength stimuli depended on the orientation of white lines superimposed on the wavelengths. Over different conditions in Experiment 1, three wavelength differences were combined with two differences between successively presented line orientations. Measures of stimulus discriminability increased with increases in the difference between both orientation and wavelength stimuli. Conditional-discrimination performance was thus conjointly determined by stimulus disparity in the successive and simultaneous discriminations. In Experiment 2, ratios of rates of reinforcement contingent upon the two categories of correct responses were varied over several conditions for difficult and easy discriminations. Ratios of responses to wavelength pairs were sensitive to variations in the reinforcement ratio to a greater extent for the more difficult orientation discrimination than for the easier orientation discrimination. Performance in the conditional discrimination was therefore determined by the interacting effects of stimulus disparity and the relative rates of reinforcement contingent upon the two correct choices. It was concluded that the effect of temporally distant reinforcement on behavior in a prevailing schedule component is attenuated to an extent that depends on similarity of stimuli that delineate the successive components.     

The Role of Atomic Repertoires in Complex Behavior

Evolution and reinforcement shape adaptive forms and adaptive behavior through many cycles of blind variation and selection, and therein lie their parsimony and power. Human behavior is distinctive in that this shaping process is commonly "short circuited": Critical variations are induced in a single trial. The processes by which this economy is accomplished have a common feature: They all exploit one or more atomic repertoires, elementary units of behavior each under control of a distinctive stimulus. By appropriate arrangements of these discriminative stimuli, an indefinite number of permutations of atomic units can be evoked. When such a permutation satisfies a second contingency, it can come under control of the relevant context, and the explicit arrangement of discriminative stimuli will no longer be required. Consequently, innovations in adaptive behavior can spread rapidly through the population. A consideration of atomic repertoires informs our interpretation of generalized operants and other phenomena that are otherwise difficult to explain. Observational learning is discussed as a case in point.     

Comparing representations and computations in single neurons versus neural networks

Single-neuron-level explanations have been the gold standard in neuroscience for decades. Recently, however, neural-network-level explanations have become increasingly popular. This increase in popularity is driven by the fact that the analysis of neural networks can solve problems that cannot be addressed by analyzing neurons independently. In this opinion article, I argue that while both frameworks employ the same general logic to link physical and mental phenomena, in many cases the neural network framework provides better explanatory objects to understand representations and computations related to mental phenomena. I discuss what constitutes a mechanistic explanation in neural systems, provide examples, and conclude by highlighting a number of the challenges and considerations associated with the use of analyses of neural networks to study brain function.     

Control of tantrum behavior by operant techniques during experimental verbal training

A technique of controlling undesirable or disruptive behavior during an ongoing program of verbal training with a retardate is described. The technique required that the stimulus materials of the verbal training program be graded according to difficulty, i.e., in terms of the length and complexity of the stimulus materials. (This resulted in an initial grading of the stimulus materials into different levels of probability of reinforcement.) Changes by the experimenter from high-difficulty to low-difficulty stimuli for two trials contingent upon disruptive behavior increased the rate of that behavior; changes from low-difficulty to high-difficulty stimuli for two trials contingent upon disruptive behavior decreased its rate. Thus, contingent alternation of the stimulus materials of the ongoing training program controlled the frequency of undesirable behaviors within the experimental sessions. This technique may comprise an alternative to other procedures which require punishment or timeout from the ongoing program.     

A modular neural-network model of the basal ganglia’s role in learning and selecting motor behaviours

This work presents a modular neural-network model (based on reinforcement-learning actor–critic methods) that tries to capture some of the most relevant known aspects of the role that basal ganglia play in learning and selecting motor behavior related to different goals. The model uses a mixture of experts network for the critic and a hierarchical network with two levels for the actor. Some simulations with the model show that basal ganglia select ‘chunks’ of behavior whose ‘details’ are specified by direct sensory-motor pathways, and how emergent modularity can help to deal with tasks with asynchronous multiple goals. A ‘top-down’ approach is adopted that first analyses some adaptive non-trivial interaction of a whole (simulated) organism with the environment, and its capacity to learn, and then attempts to implement these functions with neural architectures and mechanisms that have an empirical neuroanatomical and neurophysiological foundation.     

Sub-Optimal Choice by Pigeons: Failure to Support The Allais Paradox

Pigeons show a preference for an alternative that provides them with discriminative stimuli (sometimes a stimulus that predicts reinforcement and at other times a stimulus that predicts the absence of reinforcement) over an alternative that provides them with nondiscriminative stimuli, even if the nondiscriminative stimulus alternative is associated with 2.5 times as much reinforcement (Stagner & Zentall, 2010). In Experiment 1 we found that the delay to reinforcement associated with the nondiscriminative stimuli could be reduced by almost one half before the pigeons were indifferent between the two alternatives. In Experiment 2 we tested the hypothesis that the preference for the discriminative stimulus alternative resulted from the fact that, like humans, the pigeons were attracted by the stimulus that consistently predicted reinforcement (the Allais paradox). When the probability of reinforcement associated with the discriminative stimulus that predicted reinforcement was reduced from 100% to 80% the pigeons still showed a strong preference for the discriminative stimulus alternative. Thus, under these conditions, the Allais paradox cannot account for the sub-optimal choice behavior shown by pigeons. Instead we propose that sub-optimal choice results from positive contrast between the low expectation of reinforcement associated with the discriminative stimulus alternative and the much higher obtained reinforcement when the stimulus associated with reinforcement appears. We propose that similar processes can account for sub-optimal gambling behavior by humans.