Contingency Horizon: on Private Events and the Analysis of Behavior

Skinner’s radical behaviorism incorporates private events as biologically based phenomena that may play a functional role with respect to other (overt) behavioral phenomena. Skinner proposed four types of contingencies, here collectively termed the contingency horizon, which enable certain functional relations between private events and verbal behavior. The adequacy and necessity of this position has met renewed challenges from Rachlin’s teleological behaviorism and Baum’s molar behaviorism, both of which argue that all “mental” phenomena and terminology may be explained by overt behavior and environment–behavior contingencies extended in time. A number of lines of evidence are presented in making a case for the functional characteristics of private events, including published research from behavior analysis and general experimental psychology, as well as verbal behavior from a participant in the debate. An integrated perspective is offered that involves a multiscaled analysis of interacting public behaviors and private events.     

GAP-43 gene expression regulates information storage

Previous reports have shown that overexpression of the growth- and plasticity-associated protein GAP-43 improves memory. However, the relation between the levels of this protein to memory enhancement remains unknown. Here, we studied this issue in transgenic mice (G-Phos) overexpressing native, chick GAP-43. These G-Phos mice could be divided at the behavioral level into “spatial bright” and “spatial dull” groups based on their performance on two hidden platform water maze tasks. G-Phos dull mice showed both acquisition and retention deficits on the fixed hidden platform task, but were able to learn a visible platform task. G-Phos bright mice showed memory enhancement relative to wild type on the more difficult movable hidden platform spatial memory task. In the hippocampus, the G-Phos dull group showed a 50% greater transgenic GAP-43 protein level and a twofold elevated transgenic GAP-43 mRNA level than that measured in the G-Phos bright group. Unexpectedly, the dull group also showed an 80% reduction in hippocampal Tau1 staining. The high levels of GAP-43 seen here leading to memory impairment find its histochemical and behavioral parallel in the observation of Rekart et al. (Neuroscience 126: 579–584) who described elevated levels of GAP-43 protein in the hippocampus of Alzheimer’s patients. The present data suggest that moderate overexpression of a phosphorylatable plasticity-related protein can enhance memory, while excessive overexpression may produce a “neuroplasticity burden” leading to degenerative and hypertrophic events culminating in memory dysfunction.     

Olfactory learning in individually assayed Drosophila larvae

Insect and mammalian olfactory systems are strikingly similar. Therefore, Drosophila can be used as a simple model for olfaction and olfactory learning. The brain of adult Drosophila, however, is still complex. We therefore chose to work on the larva with its yet simpler but adult-like olfactory system and provide evidence for olfactory learning in individually assayed Drosophila larvae. We developed a differential conditioning paradigm in which odorants are paired with positive (“+” fructose) or negative (“-” quinine or sodium chloride) gustatory reinforcers. Test performance of individuals from two treatment conditions is compared—one received odorant A with the positive reinforcer and odorant B with a negative reinforcer (A+/B-); animals from the other treatment condition were trained reciprocally (A-/B+). During test, differences in choice between A and B of individuals having undergone either A+/B- or A-/B+ training therefore indicate associative learning. We provide such evidence for both combinations of reinforcers; this was replicable across repetitions, laboratories, and experimenters. We further show that breaks improve performance, in accord with basic principles of associative learning. The present individual assay will facilitate electrophysiological studies, which necessarily use individuals. As such approaches are established for the larval neuromuscular synapse, but not in adults, an individual larval learning paradigm will serve to link behavioral levels of analysis to synaptic physiology.     

Contrast and reallocation of extraneous reinforcers between multiple-schedule components

Four pigeons responded in components of multiple schedules in which two responses were available and reinforced with food. Pecks on the left key (“main” key) were reinforced at a constant rate in one component and at a rate that varied over conditions in the other component. When reinforcer rate was varied, behavioral contrast occurred in the constant component. On the right key (“extra” key), five variable-interval schedules and one variable-ratio schedule, presented conjointly, arranged reinforcers for responses in all conditions. These conjoint schedules were common to both multiple-schedule components—rather than unique to particular components—and reinforcers from these schedules could therefore be arranged in one component and obtained during the other component. In this way, the additional reinforcers were analogous to the “extraneous” reinforcers thought to maintain behavior other than pecking in conventional multiple schedules. Response rate on the extra key did not change systematically over conditions in the constant component, and in the varied component extra responding was inversely related to main-key reinforcement. All subjects obtained more extra-key reinforcers in whichever component arranged fewer main-key reinforcers. Consistent with the theory that reallocation of extraneous reinforcers may cause behavioral contrast, absolute reinforcer rate for the extra key in the constant component was low in conditions that produced positive contrast on the main key and high in those that produced negative contrast. Also consistent with this theory, behavioral contrast was reduced in two conditions that canceled extra-key reinforcers that had been arranged but not obtained at the end of components. Thus, a constraint on reallocation markedly reduced the extent of contrast.     

The Organization of Extrinsic Neurons and Their Implications in the Functional Roles of the Mushroom Bodies in Drosophila melanogaster Meigen

Although the importance of the Drosophila mushroom body in olfactory learning and memory has been stressed, virtually nothing is known about the brain regions to which it is connected. Using Golgi and GAL4–UAS techniques, we performed the first systematic attempt to reveal the anatomy of its extrinsic neurons. A novel presynaptic reporter construct, UAS-neuronal synaptobrevin–green fluorescent protein (n-syb–GFP), was used to reveal the direction of information in the GAL4-labeled neurons. Our results showed that the main target of the output neurons from the mushroom body lobes is the anterior part of the inferior medial, superior medial, and superior lateral protocerebrum. The lobes also receive afferents from these neuropils. The lack of major output projections directly to the deutocerebrum’s premotor pathways discourages the view that the role of the mushroom body may be that of an immediate modifier of behavior. Our data, as well as a critical evaluation of the literature, suggest that the mushroom body may not by itself be a “center” for learning and memory, but that it can equally be considered as a preprocessor of olfactory signals en route to “higher” protocerebral regions.     

The associative relation underlying autoshaping in the pigeon

Fifteen pigeons were exposed to either response-independent or response-dependent schedules of water reinforcement, whereby water was injected directly into the unrestrained pigeons' mandibles. Key-contact responses were released by a lighted key correlated with water, but not by a lighted key uncorrelated with water. A negative response-reinforcer contingency suppressed autoshaped key-contact responses, resulting in responding directed away from the lighted key. In all pigeons, water injected directly into the mandibles elicited a consummatory fixed-action pattern of “mumbling” and swallowing. The lighted key correlated with water released a broader set of both appetitive and consummatory responses: approach to the lighted key, “bowing”, “rooting”, “mumbling”, and swallowing. Key-contact responses were “rooting” and “mumbling” motions of the beak on the surface of the key. Views of autoshaping based on stimulus substitution or stimulus surrogation do not fully explain the origin of autoshaped responses not previously elicited by the reinforcer. The present findings are consonant with views of conditioning that emphasize the large degree of biological pre-organization in conditioned response patterns, and the importance of associative factors in the control of such patterns.     

Unbiased and unnoticed verbal conditioning: the double agent robot procedure

Subjects who were told they were “experimenters” attempted to reinforce fluent speech in a supposed subject with whom they spoke via intercom. The supposed subject was to say nouns, one at a time, on request by the “experimenter”, who reinforced fluent pronunciation with points. Actually, the “experimenter” was talking to a multi-track tape recording, one track of which contained fluently spoken nouns, the other track containing disfluently spoken nouns. If the “experimenter's” request for the next noun was in a specified form a word from the fluent track was played to him as reinforcement; requests in any other form produced the word from the disfluent track. Repeated conditioning of specific forms of requests was accomplished with two subject-“experimenters,” who were unable to describe changes in their own behavior, or the contingencies applied. This technique improved upon an earlier method that had yielded similar results, but was less thoroughly controlled against possible human bias.     

Bias in self-evaluation: Signal probability effects

Two experiments examined apparent signal probability effects in simple verbal self-reports. After each trial of a delayed matching-to-sample task, young adults pressed either a “yes” or a “no” button to answer a computer-presented query about whether the most recent choice met a point contingency requiring both speed and accuracy. A successful matching-to-sample choice served as the “signal” in a signal-detection analysis of self-reports. Difficulty of matching to sample, and thus signal probability, was manipulated via the number of nonmatching sample and comparison stimuli. In Experiment 1, subjects exhibited a bias (log b) for reporting matching-to-sample success when success was frequent, and no bias or a bias for reporting failure when success was infrequent. Contingencies involving equal conditional probabilities of point consequences for “I succeeded” and “I failed” reports had no systematic effect on this pattern. Experiment 2 found signal probability effects to be evident regardless of whether referent-response difficulty was manipulated in different conditions or within sessions. These findings indicate that apparent signal probability effects in self-report bias that were observed in previous studies probably were not an artifact of contingencies intended to improve self-report accuracy or of the means of manipulating signal probability. The findings support an analogy between simple self-reports and psychophysical judgments and bolster the conclusion of Critchfield (1993) that signal probability effects can influence simple self-reports much as they do reports about external stimuli in psychophysical experiments.     

The effectiveness of fading in programming a simultaneous form discrimination for retarded children

A non-verbal teaching program, combined with reinforcement and extinction (Program Group), was compared with reinforcement and extinction alone (Test Group) in teaching retarded children to discriminate circles from ellipses. In the Program Group, fading techniques were used to transfer stimulus control from “bright vs. dark” to “form vs. no-form” and then to “circle vs. ellipse”. The Test Group had the task of learning the circle-ellipse discrimination with no prior teaching program. With the program, seven of 10 children learned the circle-ellipse discrimination. Without the program, one of nine learned. The eight Test-Group children who failed to learn circle vs. ellipse were then given the opportunity to learn the form no-form discrimination by reinforcement and extinction alone, without fading. Six of the eight learned, but only three of these six then learned circle vs. ellipse on a second test. All seven Program-Group children who had learned form vs. no-form also learned the circle-ellipse discrimination by means of fading; each of the seven made fewer errors than any of the three who succeeded on the second test. Children who failed to learn circle vs. ellipse adopted response patterns incompatible with the development of appropriate stimulus control.     

Repeated acquisition in the analysis of rule-governed behavior

Five children, ranging in age from 3½ years to 5½ years, were taught various four-response chains using conditioned reinforcement. Experiment 1 investigated the effects of presenting “instruction” stimuli—a sequence of lights over the correct response buttons—to assess their role in facilitating the acquisition of a chain of responses. Without the “instruction” stimuli, children made many errors before responses were brought under the control of the programmed contingencies. When confronted with the same contingencies later in the day, these subjects made fewer errors. In contrast, in the presence of the “instruction” stimuli, subjects made virtually no errors. However, when the “instruction” stimuli were discontinued in the subsequent session, all 5 subjects made errors. In Experiment 2, the subjects were taught to verbalize the contingencies during the phase without the “instruction” stimuli. This resulted in errorless performance during the subsequent exposure to the same procedure, but errors nevertheless occurred again during reexposure to the procedure with the “instruction” stimuli discontinued.     

The effects of punishment intensity on squirrel monkeys

Responses of squirrel monkeys were maintained by a variable-interval schedule of food reinforcement. Concurrently, punishment consisting of a brief electric shock followed each response. As has been found for pigeons and rats, punishment did not produce extreme, all-or-none reactions. By gradually increasing the punishment intensity it was possible to produce response rates intermediate to no suppression and complete suppression. Similarly, the moment-to-moment response rate was free of extreme fluctuations. A “warm-up” effect occurred in which the punished responses were especially suppressed during the initial part of a session. The pre-punishment performance was negatively accelerated within a session, and punishment reduced the degree of negative acceleration. When punishment was discontinued, responding recovered immediately except when suppression had been complete or prolonged. When the punishment intensity was decreased gradually, more suppression resulted at a given intensity than when intensity was increased gradually. This suggests a “behavioral inertia” effect wherein behavior at a new punishment intensity is biased toward the behavior at the previous value. A corollary generalization is that the larger the change in intensity, the less the behavior at the new value will be biased toward the behavior at the previous value.     

Plasticity of the human auditory cortex induced by discrimination learning of non-native,mora-timed contrasts of the Japanese language

In this magnetoencephalographic (MEG) study, we examined with high temporal resolution the traces of learning in the speech-dominant left-hemispheric auditory cortex as a function of newly trained mora-timing. In Japanese, the “mora” is a temporal unit that divides words into almost isochronous segments (e.g., na-ka-mu-ra and to-o-kyo-o each comprises four mora). Changes in the brain responses of a group of German and Japanese subjects to differences in the mora structure of Japanese words were compared. German subjects performed a discrimination training in 10 sessions of 1.5 h each day. They learned to discriminate Japanese pairs of words (in a consonant, anni–ani; and a vowel, kiyo–kyo, condition), where the second word was shortened by one mora in eight steps of 15 msec each. A significant increase in learning performance, as reflected by behavioral measures, was observed, accompanied by a significant increase of the amplitude of the Mismatch Negativity Field (MMF). The German subjects' hit rate for detecting durational deviants increased by up to 35%. Reaction times and MMF latencies decreased significantly across training sessions. Japanese subjects showed a more sensitive MMF to smaller differences. Thus, even in young adults, perceptual learning of non-native mora-timing occurs rapidly and deeply. The enhanced behavioral and neurophysiological sensitivity found after training indicates a strong relationship between learning and (plastic) changes in the cortical substrate.     

The role of contingencies and "principles of behavioral variation" in pigeons' pecking

Staddon and Simmelhag's proposal that behavior is produced by “principles of behavioral variation” instead of contingencies of reinforcement was tested in two experiments. In the first experiment pigeons were exposed to either a fixed-interval schedule of response-contingent reinforcement, an autoshaping schedule of stimulus-contingent reinforcement, or a fixed-time schedule of noncontingent reinforcement. Pigeons exposed to contingent reinforcement came to peck more rapidly than those exposed to noncontingent reinforcement. Staddon and Simmelhag's “principles of behavioral variation” included the proposal that patterns (interim and terminal) were a function of momentary probability of reinforcement. In the second experiment pigeons were exposed to either a fixed-time or a random-time schedule of noncontingent reinforcement. Pecking showed a constant frequency of occurrence over postfood time on the random-time schedule. Most behavior showed patterns on the fixed-time schedule that differed in overall shape (i.e., interim versus terminal) from those shown on the random-time schedule. It was concluded that both the momentary probability of reinforcement and postfood time can affect patterning.     

Magnitudes of score differences produced within sessions in a cooperative exchange procedure

Interest centered on maximal score differences produced within sessions during two-party exchange. Subjects chose between earning money independently or through potentially higher-paying exchange. In the exchange option, only one person could produce points for the other on a trial. Because each exchange response (“give”) required the giver to forego earning points independently, the larger the score difference produced (i.e., the further ahead in earnings the other person was put), the greater the reduction in the giver's earnings if the other person did not reciprocate. Results showed that scores were usually equal at the end of each session, and that subjects maintained close equality of scores throughout each session. When a response-cost contingency that punished the alternation of giving was introduced, however, large within-session score differences developed. These large differences continued to be produced after the response-cost contingency was removed. Finally, when subjects were told that the session could end at any moment, score differences were sharply reduced, indicating that production of score differences remained under the control of discriminative stimuli associated with the likelihood of reciprocation. The study suggests that with appropriate procedures, an experimental analysis of behavioral phenomena associated with the concept of “trust” may be possible.     

Recognition by the pigeon of stimuli varying in two dimensions

Pigeons served in four experiments, each of which involved about 44,000 discrete 1.2-sec trials under steady-state conditions. The first experiment scaled a short segment of the visual wavelength continuum; this dimension was then combined in a conditional discrimination with each of three others; time after reinforcement, tone frequency, and line tilt. In the two-stimulus experiments, the birds' responses were reinforced in the presence of only one stimulus combination: “582 nm” together with “2 min after reinforcement”, “3990 Hz”, or “vertical line”. Many other stimulus combinations also appeared equally often and went without reinforcement. The wavelength stimuli conformed to an equal-interval scale, and per cent response was generally linear with wavelength, when scaled on cumulative normal coordinates. The components of the compound stimulus were found to interact in a multiplicative fashion; when one component differed greatly from its reinforcement value, changes in the other component had relatively little effect. For the “time”-“wavelength” compound, this interaction appeared to be modified by the effects of set or attention. Certain response latency data are reported, and other combination rules are discussed.     

Pigeons'' spatial memory: II. Acquisition of delayed matching of key location and transfer to new locations

Five hungry pigeons first received delayed matching of key location training. Trials began with a “ready” stimulus (brief operation of the grain feeder). Then one (randomly chosen) of a set of four keys from a three-by-three matrix was lit briefly as the sample. After a short delay (retention interval), the sample key was lit again along with one of the other eight keys. A peck at the key that had served as the sample produced grain reinforcement, whereas a peck to the other key produced only the intertrial interval. After delayed matching of key location was learned, the remaining five key locations were introduced as samples. Four of the five birds performed at considerably above-chance levels on the novel sample trials during the first as well as subsequent sessions. These results suggest that pigeons sometimes learn the single rule—“choose the location that matches the sample.” The relevance of these results to the issue of whether pigeons learn a generalized matching rule (i.e., a concept of “sameness”) is discussed.     

Stimulus control of avoidance behavior

The introduction of a warning signal preceding shocks greatly increased the effectiveness of avoidance responding. Periods of “warm-up” at the beginning of the session were eliminated, and the number of shocks received by the subjects was greatly reduced. With response-shock interval constant, response rate increased as the interval between the response and the onset of the warning signal was shortened. The response tended to occur shortly after the onset of the warning signal regardless of the duration of these “safe” periods. A greatly elevated response rate was maintained even when the duration of the safe period was reduced to 0.3 sec. Thus, the pre-shock signal obtained nearly exclusive control of the responding and overrode the usual “temporal discrimination” of the response-shock interval.     

Neural coding of reward magnitude in the orbitofrontal cortex of the rat during a five-odor olfactory discrimination task

The orbitofrontal cortex (OBFc) has been suggested to code the motivational value of environmental stimuli and to use this information for the flexible guidance of goal-directed behavior. To examine whether information regarding reward prediction is quantitatively represented in the rat OBFc, neural activity was recorded during an olfactory discrimination “go”/“no-go” task in which five different odor stimuli were predictive for various amounts of reward or an aversive reinforcer. Neural correlates related to both actual and expected reward magnitude were observed. Responses related to reward expectation occurred during the execution of the behavioral response toward the reward site and within a waiting period prior to reinforcement delivery. About one-half of these neurons demonstrated differential firing toward the different reward sizes. These data provide new and strong evidence that reward expectancy, regardless of reward magnitude, is coded by neurons of the rat OBFc, and are indicative for representation of quantitative information concerning expected reward. Moreover, neural correlates of reward expectancy appear to be distributed across both motor and nonmotor phases of the task.     

Visual object recognition is facilitated by temporal community structure

Humans and others primates are highly attuned to temporal consistencies and regularities in their sensory environment and learn to predict such statistical structure. Moreover, in several instances, the presence of temporal structure has been found to facilitate procedural learning and to improve task performance. Here we extend these findings to visual object recognition and to presentation sequences in which mutually predictive objects form distinct clusters or “communities.” Our results show that temporal community structure accelerates recognition learning and affects the order in which objects are learned (“onset of familiarity”).

Our understanding of the world is grounded in sensory experience. Typically, this experience consists of contiguous streams of sensations that are richly structured in both time and space (Schapiro and Turk-Browne 2015). Such statistical structure may involve simple correlations of pairs of sensory events or, more generally, clusters of correlations between mutually predictive events forming a “temporal community” (Schapiro et al. 2013). Both humans and other primates (Miyashita 1988) can learn to predict such statistical regularities in space and time (Fiser and Aslin 2001, 2002). Moreover, statistical structure can be exploited explicitly or implicitly to enhance task performance. For example, predictable presentation order can facilitate motor learning (Kahn et al. 2018), language learning (Saffran et al. 1996), visual search (Chun and Jiang 1998; Jiang and Wagner 2004; Sisk et al. 2019), and conditional associative learning (Hamid et al. 2010).In general, implicit (unsupervised) learning of temporal structure is thought to provide a biological basis for important cognitive functions, including the formation of episodic memories, learning of task-sets, model-based planning, and structural learning (e.g., Kemp and Tenenbaum 2008; Rigotti et al. 2010; Gershman 2017; Russek et al. 2017). To improve experimental access to these phenomena, we sought behavioral evidence for interactions between learning at different hierarchical levels, namely, learning of individual objects and learning of the temporal context in which such objects are experienced.Sequences of visual presentations may exhibit different kinds of temporal structure arising from sequential dependencies. A simple kind of structure is sequential dependency between consecutively presented items (i.e., an increased probability of item X, given preceding item Y). A more complex kind of structure arises when sequential dependencies are clustered within subsets of items. This leads to longer-term dependencies (i.e., an increased probability of item X, given recent item Z) and extended sequences of items that are mutually predictive (Schapiro et al. 2013; Karuza et al. 2017; Kahn et al. 2018).The mechanisms of visual object recognition have been studied extensively (Wallis and Bülthoff 1999) with considerable evidence supporting “feature-based mechanisms” that represent three-dimensional objects in terms of multiple two-dimensional features/views (plus interpolations) (Bülthoff and Edelman 1992). Presumably, temporal regularities arise naturally in handling three-dimensional objects and help associate distinct two-dimensional views and/or features (Wallis and Bülthoff 1999). For example, when nonhuman primates learn to categorize initially unfamiliar objects, they readily form neural representations for arbitrary two-dimensional features that are diagnostic for category (Sigala and Logothetis 2002; Sigala et al. 2002). Interestingly, such representations automatically encompass predictive sequential dependencies between successive trials, even when its diagnostic information is redundant (Miyashita 1988; Wallis 1998).The effect of sequential dependencies between successive trials on visual object recognition was investigated by two previous studies, which found a reaction time advantage (Barakat et al. 2013) and a recognition memory advantage (Otsuka and Saiki 2016) for target objects that consistently follow particular objects, compared with target objects that follow varying objects. Here we extended these findings in two ways: First, we monitored the formation of recognition memory more closely and comprehensively (every presentation of every object), and second, we considered the effect of clustered dependencies creating “temporal communities” of objects (which are typically experienced for nine successive presentations).We investigated performance of observers in a visual object recognition learning task under three conditions: (1) “strongly structured” sequences comprising distinct temporal communities (clusters of mutually predictive objects), (2) “weakly structured” sequences with uniform sequential dependence, and (3) “random” or “unstructured” sequences without sequential dependence. All sequences were generated as random walks on graphs of n = 15 distinct objects (Fig. 1A), in which nodes represented distinct objects and edges represented possible transitions (in both directions). As one sequence comprised 180 object presentations, each graph was traversed multiple times (∼11.3 times). Graphs were either modular and sparsely connected (“strongly structured” sequences), or nonmodular and sparsely connected (“weakly structured” sequences), or nonmodular and fully connected (“unstructured” or “random” sequences). In “strongly structured” sequences, approximately 9.2 ± 0.1 successive presentations (mean ± SEM) featured objects of the same temporal community.Open in a separate windowFigure 1.Presentation sequence and trial structure. (A) Presentation sequences were generated as (nearly) random walks on three types of graphs, with nodes representing a distinct object and edges representing possible transitions (in both directions). A sparsely connected, modular graph generated “strongly structured” sequences with distinct community structures (left), a sparsely connected, nonmodular graph generated “weakly structured” sequences (middle), and a full connected graph generated “unstructured” or “random” sequences (right). (B) Presentation sequences consisted of 180 complex, three-dimensional objects (shown rotating for 2 sec about a randomly oriented axis in the frontal plane). Of these, 170 ± 0.04 (mean ± SEM) objects were recurring, and 9.2 ± 0.04 objects were nonrecurring. Observers categorized each object as “familiar” or “unfamiliar.” Over the four sessions of 1 wk, observers performed 24 runs and viewed 4320 presentations, with every recurring object appearing at least 250 times.One presentation sequence (“run”) comprised exactly 180 objects and on average included 9.2 ± 0.04 (mean ± SEM) nonrecurring objects appearing exactly once during the entire experiment. Nonrecurring objects were spaced 14–19 presentations apart. The remaining 170 ± 0.04 objects were recurring and were selected by performing a pseudorandom walk on a graph (Fig. 1A), albeit with some restrictions: no direct repeats and returns were permitted (e.g., X–X or X–Y–X) and all n = 15 objects were repeated comparably often (11.4 ± 0.04 repetitions). The repetition latency for any given object ranged from three to >60 presentations. Very short latencies (of three to five presentations) were far more common in strongly structured sequences than in weakly structured or unstructured sequences (Supplemental Fig. S8).To control the difficulty of shape recognition, ensure initial unfamiliarity of all objects, and minimize interference from semantic associations, we generated complex three-dimensional objects by convolving two closed Bezier curves in a plane. Complexity was controlled by number and the position of random seeds for the two curves. The pairwise dissimilarity of the resulting complex objects was statistically unrelated to their pairwise distance in the presentation sequence (see Supplemental Fig. S1). To ensure this, dissimilarity was quantified in terms of the vector distance between depth maps (of resolution 64 × 64 × 64) obtained from six viewing directions along the three principal component axes.Objects were presented for 2 sec rotating with an angular velocity of 144 deg/sec about an axis in the frontal plane. Starting angle and axis orientation were randomized for each trial, forcing observers to become familiar with the full three-dimensional shape (rather than just certain features). Presentation periods were separated by 0.5-sec transition periods, during which the previous object disappeared toward a distant location on the right, while the next object approached from a distant location on the left. This was intended to encourage observers to imagine a spatially extended sequence of distinct objects (Supplemental_ Movie_S1).Twenty healthy observers (eight males and 12 females, aged 25 to 34 yr old) participated in three experiments. Two experiments compared “strongly structured” and “unstructured” sequences, and one experiment compared “strongly structured” and “weakly structured” sequences. All observers had normal or corrected to normal vision and were paid for their participation. Ethical guidelines of the Centre for Neuroscientific Innovation and Technology, Magdeburg, were followed.In order to monitor the progress of recognition learning as closely as possible, observers were required to classify every object presented as either “familiar” (seen previously) or “unfamiliar” (never seen previously). For each observer, a fresh set of 30 pairwise dissimilar objects was generated. The set was divided arbitrarily into two subsets of 15 objects, one used for “structured” sequences and the other for “unstructured” sequences. In addition, we generated a larger number (∼500) of nonrecurring objects, which appeared exactly once during the entire experiment. During each trial, the observer categorized the current object as “familiar,” “unfamiliar,” and “not sure,” by pressing a key. No feedback was provided. Observers performed this task on four different days within 1 wk, with six sequences per day (24 sequences overall). Accordingly, observers viewed 4320 presentations during which every recurring object appearing at least 250 times. After pausing for a week, observers repeated the experiment with entirely new objects and with sequences generated from another graph (Fig. 1B). Observers were told that each condition used new objects that were never shown before. To further emphasize this point, object color changed between conditions. The order of conditions (structured or unstructured) was counter-balanced between observers. Observer instructions did not mention presentation order (sequence structure).At the end of each week of testing, observers were required to additionally perform a validation task, to assess the extent to which objects had become familiar (Supplemental_Movie_S2; Supplemental_Material). In this task, observers viewed for 30 sec an array of 12 simultaneously rotating objects, of which three were randomly selected from the 15 “recurring” objects and nine objects were entirely new (never seen before). Observers were asked to pick out the three most “familiar” objects and received binary feedback (“all correct” or “one or more incorrect”). All observers approached ceiling performance (proportion correct >0.95) in all conditions (all sequence structures), confirming that almost all recurring objects had become familiar.To establish the progress of recognition learning, we analyzed 250 repetitions (over four sessions and 24 sequences) of every recurring object. To this end, we considered “sliding windows” with N_w = 5 successive presentations of a given object (for details see Supplemental Fig. S3). Note that some windows bridged successive presentation sequences and/or sessions. For each window and “recurring” object, we computed the proportion of “familiar” responses (“hit rate”) (Fig. 2A). As “familiar” objects were common, some false positives were to be expected. To take this into account, we also established a “false alarm rate” for each session, as the fraction of “nonrecurring” objects not categorized as “unfamiliar” (Fig. 2B). Combining hit rate (of a window) with false alarm rate (of the concomitant session), we performed a simplified sensitivity analysis (Macmillan and Creelman 2004) to obtain a corrected classification performance ρ and decision bias b for each window and “familiar” object (see the Supplemental_Material). Alternative sensitivity analyses and performance measures (A′, d′; Stanislaw and Todorov 1999) did not materially alter the results.Open in a separate windowFigure 2.Time course of recognition learning. (A) Average hit rate (recurring categorized as familiar, per window) increases with the number of presentations of a given object. (B) Average false alarm rate (nonrecurring not categorized as unfamiliar, per session) decreases with the number of presentations. (C) Average corrected performance ρ increases nearly monotonically with presentation number. It was consistently larger for strongly structured sequences (with temporal community structure) than for unstructured sequences. (D) Average criterion bias b, as a function of presentation number. Green regions indicate the transition between sessions (20%–80% of objects in previous session).The resulting corrected performance ρ (mean and SEM, assuming binomial variability) is shown in Figure 2C. Performance increased nearly monotonically, but was consistently superior when objects were presented with “strongly structured” sequences with “temporal community structure” than when they were presented in unstructured sequences. This difference was significant after ∼60 presentations. As expected, observers rapidly developed a liberal bias (favoring “familiar” responses), which weakened somewhat over subsequent sessions (Fig. 2D).We also analyzed the time-development of average response times (RTs). Consistent with the performance results, RTs decreased faster for strongly structured sequences than for unstructured sequences (Supplemental_Material; Supplemental Fig. S2).In addition to the gradual increase in the probability of recognizing recurring objects, we also sought to determine the point in time at which individual objects became familiar (“onset of familiarity”). We defined this point in two alternative ways: (1) as the first window in which corrected performance exceeded a threshold of ρ ≥ 0.875 (high threshold approach) or (2) as the window in which entropy Hρ=−[ρlog2(ρ)+(1−ρ)log2(1−ρ)] of corrected performance reached its peak value (low threshold approach). Note that entropy peaks at the transition from exclusively “unfamiliar” to exclusively “familiar” responses.After establishing the “onset of familiarity” for each object, we ranked all objects by order of onset and established the “onset separation” between object pairs in terms of onset rank (Δn) and presentation rank (Δk). The median separation of successive onsets (defined by threshold or entropy) was nine or 16 presentations, respectively. Interestingly, the median separation of successive onsets in same cluster was roughly thrice as long, with 24 and 50 presentations, respectively, implying that successive onsets occurred during separate visits to a given community.In strongly structured sequences, one may distinguish objects pairs XY that are “adjacent” [follow each other with P(Y|X) = 0.25] or “nonadjacent” [never follow each other, P(Y|X) = 0]. In addition, one may distinguish object pairs within the same community (either adjacent or nonadjacent) and between different communities (also either adjacent or nonadjacent). Note that the objects linking different communities (“linking objects”) contribute both “adjacent” pairs in different communities and “nonadjacent” pairs in the same community (Fig. 3B). We analyzed the “onset of familiarity” for different object pairs (as defined above), specifically, the probability that the two members of a pair exhibit successive onsets (Δn = 1) or nearly successive (Δn = 2) onsets. Interestingly, the probability of successive onsets was significantly higher than chance for objects in the same community (null hypothesis H₀: “onsets” are ordered randomly) (Fig. 3A). Moreover, we found the probability of successive “onsets” to be significantly elevated for “adjacent” objects in the same cluster, insignificantly elevated for “adjacent” objects in different clusters (“linking objects”), and significantly reduced for “nonadjacent” objects in different clusters (P < 0.05; corrected for false discovery rate of multiple comparisons) (Fig. 3B; Benjamini and Hochberg 1995).Open in a separate windowFigure 3.Analysis of the onset of familiarity with individual objects. (A) Successive onsets of familiarity (Δn = 1) are far more likely ([**] P < 0.005) for objects within the same cluster than would be expected by chance (dashed line). For nearly successive onsets (Δn = 2) this effect was not observed. (B) Comparison of frequency of successive onsets, compared with chance level, for objects pairs either in the same cluster (outlined blue and cyan) or in different clusters (green and red), which are either adjacent (blue and green) or nonadjacent on the graph (cyan and red). Frequency is significantly elevated ([*] P < 0.05 FDR corrected) for adjacent objects in the same cluster (blue) and suppressed for nonadjacent objects in different clusters (red).We conclude that temporal community structure had a significant effect on the order of recognition learning in the sense that familiarity of one object in a community facilitated familiarity of another object in the same community, provided the latter was “adjacent” [i.e., followed the former sometimes, P(Y|X) = 0.25]. Interestingly, no such “domino effect” was observed for the objects linking two different communities (i.e. adjacent objects in different communities).The results presented in Figures 2 and ​and33 were replicated with an additional eight observers in a second experiment of almost identical design (Supplemental Figs. S4, S6).To dissociate the effects of cluster-membership and adjacency, we also conducted a third experiment, in which six further observers viewed either “weakly structured” presentation sequences (during 1 wk) or “strongly structured” sequences (during another week). To generate “weakly structured” sequences without temporal communities, we generated sparsely connected graphs with exactly four links per node, but without any triangular link formations (Maslov and Sneppen 2002; Rubinov and Sporns 2010). Recognition learning was faster for “strongly structured” sequences than for “weakly structured” ones. The “domino” effect described above was again observed for “strongly structured” sequences (with both “onset” definitions), but to some extent also for “weakly structured” sequences (for one “onset” definition). Thus, the ordering of “onsets” of familiarity may be affected both by community membership and by adjacency in the presentation sequence (Supplemental Figs. S5, S7).In this study, we investigated the effect of temporal community structure by comparing more or less structured presentation sequences. First, in “weakly structured” sequences, sparse connectivity of the generative graph ensured that each object predicted the next object with 25% probability (one of four possibilities). Second, in “strongly structured” sequences, the (equally sparse) generative graph was clustered into three communities of five objects, so that each object predicted the community membership of the next object with 90% probability (18 of 20 possibilities).Previous studies of statistical learning did not aim to closely follow the learning of individual items (Siegelman et al. 2018). Here we sought to monitor the degree of familiarity of each individual object over successive presentations (Fig. 2). Whereas classification performance improved monotonically with presentation number for all sequences, a significant performance advantage developed quickly (over 60 to 70 presentations) for “strongly structured” sequences compared with either “unstructured” or “weakly structured” sequences (Supplemental Fig. S5). Note that recognition performance improved comparably over time, with or without having practiced stimulus-response mapping in a separate training session (experiments 2 and 3). Accordingly, we do not believe that motor learning contributed appreciably to these results.Thanks to close monitoring, we could almost always determine the onset of familiarity for an individual object. Interestingly, the ordering of onsets did not appear to be fully random, in that objects of the same community (“temporal community”) tended to become familiar after one another more often than expected by chance. Interestingly, this “domino effect” typically did not occur within one “extended visit” to a community but over subsequent visits to a given community. This “domino effect” was particularly pronounced for adjacent objects in the same community, but was not observed for adjacent objects in different communities. As a similar effect was observed for adjacent objects in “weakly structured” sequences without communities, there seems to be a contribution of frequent temporal proximity.At the end of training, all objects had become familiar and could be retrieved explicitly from long-term memory, for both structured and unstructured sequences. The reason for the observed difference in learning rates remains unclear. One possibility is that structured sequences pose a reduced working-memory load, facilitating encoding and accelerating learning. When large sets of items are divided (“chunked”) into subsets, both chunked and nonchunked items benefit and are learned more readily. Presumably, chunking reduces the dimensionality of the classification problem presented by each item (just like chunking the search array in an odd-man-out task reduces the dimensionality of target detection). This reduced dimensionality could then lower working-memory load and facilitate classification by comparison with long-term memory for both familiar (chunked) items and unfamiliar (nonchunked) items. Another important factor might be that temporal communities reduce repetition latencies (Supplemental Fig. S8). There is evidence that timely repetitions help consolidate memories, whereas delayed repetitions leave memories prone to disruption (Thalmann et al. 2019).Previous studies of the effect of “temporal community structure” have shown that cluster borders are detectable (Schapiro et al. 2013) and that such borders elevate reaction time (Kahn et al. 2018; Karuza et al. 2019). As border items are thought to facilitate encoding/retrieval (Swallow et al. 2009), one might have expected accelerated recognition learning for “linking objects” that join two different clusters. However, in our paradigm, neither learning rate nor ordering of onsets of familiarity distinguished “linking objects” from other objects. In fact, our results suggest that any chunking benefits (Thalmann et al. 2019) apply more to objects within clusters than to objects that “link” clusters.In summary, we showed that the presence of temporal communities of mutually predictive objects accelerates recognition learning for complex, three-dimensional objects and alters the order of recognition learning such that members of a group are often learned after one another (but separated by many intervening presentations).     

Preference for signaled over unsignaled shock schedules: Ruling out asymmetry and response fixation as factors

Experiment 1 tested whether a “symmetrical” choice procedure yields results different from those previously reported using the “unidirectional” standard changeover procedure (e.g., Badia & Culbertson, 1972). Subjects could change at any time from unsignaled to signaled shock by pressing a lever and from signaled to unsignaled shock by pressing a second lever. Results were identical to those of the standard procedure and showed that the standard procedure is fully adequate. Experiment 2 tested whether choice of high density signaled shock over low-density unsignaled shock (Badia, Coker, & Harsh, 1973) resulted from initial training with equal-density schedules. Subjects were trained and tested with signaled shock twice as dense as unsignaled shock. Three of four subjects strongly preferred the signaled condition, thus ruling out carry-over and “response fixation” as alternative explanations.