When senses meet functions: an amodal stage in conceptual processing

Research has shown that concepts are associated with modality-specific features widely distributed in the brain and that semantic hubs or convergence zones may exist to mediate or integrate distributed representations. Although much neuropsychological and neuroimaging literature has provided support for the existence of hubs/convergence zones from the spatial perspective, opinions differ as to whether the representations in the hubs/convergence zones are modality-specific or amodal. The current ERP study aimed to explore whether there is an amodal stage in conceptual processing from the temporal perspective. Participants were presented with prime-target pairs and were asked to judge whether the prime (a sensory or functional feature) could be an attribute of the target (a noun in a natural or artifactual category). The results revealed that a semantic hub or convergence zone may be reached no later than 350–450?ms in conceptual processing and that the representations in the hub may be amodal.     

Long-distance feedback projections to area V1: Implications for multisensory integration,spatial awareness,and visual consciousness

It is generally agreed that information flow through the cortex is constrained by a hierarchical architecture. Recent experimental evidence suggests that projections descending the hierarchy and targeting the primary visual cortex (area V1) may play an essential role in perceptual processes. We have, therefore, reexamined feedback projections to area V1, using retrograde tracer injections in this area. In addition to well-known areas, quantification of labeling in higher cortical areas reveals a number of hitherto unknown long-distance feedback connections originating from auditory (A1), multisensory (STP) cortices, but also from a perirhinal area (36). These feedback projections from advanced cortical stations, a global feature shared by areas that belong to the ventral visual stream, could play an important role in early multisensory integration and spatial awareness and could provide the physical substrate for the involvement of area V1 in visual consciousness.     

Dissociated verbal and nonverbal retrieval and learning following left anterior temporal damage

A subject with circumscribed damage to mesial and nonmesial left anterior temporal structures (which included the left entorhinal cortex and hippocampus and the left anterolateral association cortices) maintains accurate nonverbal records of past experiences, and acquires accurate new nonverbal records. However, he has a severe defect in the retrieval and acquisition of specific verbal tags denoting those records. This dissociation is revealed by standard neuropsychological tasks, and by experimental probes focused on (1) retrieval of previously learned verbal and nonverbal entities; and (2) acquisition of new visuoverbal relationships. The results extend our current understanding of the manner in which mesial and nonmesial structures in the anterior temporal lobe contribute to the formation and storage of verbal and nonverbal records.     

The Developmental Trajectory from Amodal Perception to Empathy and Communication: The Role of Mirror Neurons in This Process

In a recent paper entitled “Mirror Neurons. Procedural Learning and the Positive New Experience” (Wolf et al., 2000), data were presented about a special type of neuron, the mirror neuron, originally located by Rizzolatti and his colleagues (1995). These neurons were discussed as they related to a particular developmental view of psychoanalysis, developmental systems self psychology (Shane, Shane, and Gales. 1997).

In this paper, we focus on how this mirror neuron system might contribute to the development of communicative abilities in humans. First we summarize the research findings about mirror neurons and how they apply to humans. We then attempt to demonstrate how the mirror neuron system might be involved in a developmental sequence hypothesized by Kohut (1984), Stern (1985), and others to begin in infancy. We postulate that this trajectory starts with the onset of “amodal perception” (Stern, 1985) and then proceeds to affect resonance, joint attention, and ultimately to symbolization of language. In this paper, we attempt to integrate these concepts with a formulation of empathy and demonstrate what might go awry in developmental disorders when the normative sequence of development described above does not take place.     

Visual cognition influences early vision: the role of visual short-term memory in amodal completion

A partly occluded visual object is perceptually filled in behind the occluding surface, a process known as amodal completion or visual interpolation. Previous research focused on the image-based properties that lead to amodal completion. In the present experiments, we examined the role of a higher-level visual process-visual short-term memory (VSTM)-in amodal completion. We measured the degree of amodal completion by asking participants to perform an object-based attention task on occluded objects while maintaining either zero or four items in visual working memory. When no items were stored in VSTM, participants completed the occluded objects; when four items were stored in VSTM, amodal completion was halted (Experiment 1). These results were not caused by the influence of VSTM on object-based attention per se (Experiment 2) or by the specific location of to-be-remembered items (Experiment 3). Items held in VSTM interfere with amodal completion, which suggests that amodal completion may not be an informationally encapsulated process, but rather can be affected by high-level visual processes.     

Neural systems behind word and concept retrieval

Using both the lesion method and functional imaging (positron emission tomography) in large cohorts of subjects investigated with the same experimental tasks, we tested the following hypotheses: (A) that the retrieval of words which denote concrete entities belonging to distinct conceptual categories depends upon partially segregated regions in higher-order cortices of the left temporal lobe; and (B) that the retrieval of conceptual knowledge pertaining to the same concrete entities also depends on partially segregated regions; however, those regions will be different from those postulated in hypothesis A, and located predominantly in the right hemisphere (the second hypothesis tested only with the lesion method). The analyses provide support for hypothesis A in that several regions outside the classical Broca and Wernicke language areas are involved in name retrieval of concrete entities, and that there is a partial segregation in the temporal lobe with respect to the conceptual category to which the entities belong, and partial support for hypothesis B in that retrieval of conceptual knowledge is partially segregated from name retrieval in the lesion study. Those regions identified here are seen as parts of flexible, multi-component systems serving concept and word retrieval for concrete entities belonging to different conceptual categories. By comparing different approaches the article also addresses a number of method issues that have surfaced in recent studies in this field.     

Infants' perceptual differentiation of amodal and modality-specific audio-visual relations.

Ninety-six infants of 3 1/2 months were tested in an infant-control habituation procedure to determine whether they could detect three types of audio-visual relations in the same events. The events portrayed two amodal invariant relations, temporal synchrony and temporal microstructure specifying the composition of the objects, and one modality-specific relation, that between the pitch of the sound and the color/shape of the objects. Subjects were habituated to two events accompanied by their natural, synchronous, and appropriate sounds and then received test trials in which the relation between the visual and the acoustic information was changed. Consistent with Gibson's increasing specificity hypothesis, it was expected that infants would differentiate amodal invariant relations prior to detecting arbitrary, modality-specific relations. Results were consistent with this prediction, demonstrating significant visual recovery to a change in temporal synchrony and temporal microstructure, but not to a change in the pitch-color/shape relations. Two subsequent discrimination studies demonstrated that infants' failure to detect the changes in pitch-color/shape relations could not be attributed to an inability to discriminate the pitch or the color/shape changes used in Experiment 1. Infants showed robust discrimination of the contrasts used.     

Redundant amodal properties facilitate operant learning in 3-month-old infants

The current study examined the role redundant amodal properties play in an operant learning task in 3-month-old human infants. Prior studies have suggested that the presence of redundant amodal information facilitates detection and discrimination of amodal properties and potentially functions to influence general learning processes such as associative conditioning. The current study examined how human infants use redundant amodal information (visual and haptic) about the shape of an object to influence learning of an operant response. Infants learned to kick to move a mobile of cylinders while either holding a cylinder, a rectangular cube, or no object. Kick rate served as the dependent measure. The results showed that infants given matching redundant amodal properties (e.g., viewed cylinders while holding a cylinder) showed facilitated operant learning whereas infants given mismatching redundant amodal properties showed inhibited operant learning. These results support and extend the Intersensory Redundancy Hypothesis by demonstrating that amodal redundancy influences complex learning processes such as operant conditioning.     

Visible persistence following a brief increment in stimulus luminance

In previous studies, it has been demonstrated that visible persistence--the period for which the perceived duration of a stimulus exceeds its physical duration--can be extended by briefly incrementing the luminance of the stimulus immediately prior to offset. Using a two-component pattern integration task, we show that this effect is not an artifact of change in the total luminous flux within the stimulus. Visible persistence was unaffected by overall luminance of the stimulus. It was also time-locked to the luminance increment. Visible persistence is seen to result from a process that is initiated by stimulus onset and that can be either wholly or partially reinitiated by the onset of the luminance increment. The duration of this process (which determines the duration of stimulus visibility) can be modified in a graded fashion by stimulus events that occur after its initiation. We outline a single-process inhibitory feedback model of the persistence mechanism that accounts for the present findings.     

Neuronal assemblies: necessity, signature and detectability

The ease with which highly developed brains can generate representations of a virtually unlimited diversity of perceptual objects indicates that they have developed very efficient mechanisms to analyse and represent relations among incoming signals. Here, we propose that two complementary strategies are applied to cope with these combinatorial problems. First, elementary relations are represented by the tuned responses of individual neurons that acquire their specific response properties (feature selectivity) through appropriate convergence of input connections in hierarchically structured feed-forward architectures. Second, complex relations that cannot be represented economically by the responses of individual neurons are represented by assemblies of cells that are generated by dynamic association of individual, featureselective cells. The signature identifying the responses of an assembly as components of a coherent code is thought to be the synchronicity of the respective discharges. The compatibility of this hypothesis is examined in the context of recent data on the dynamics of synchronization phenomena, the dependence of synchronization on central states and the relations between the synchronization behaviour of neurons and perception.     

Neural correlates of partial transmission of sensorimotor information in the cerebral cortex

Using single neuron recordings in monkey primary motor (MI) cortex, two series of experiments were conducted in order to know whether response preparation can begin before perceptual processing finishes, thus providing evidence for a temporal overlap of perceptual and motor processes.

In Experiment 1, a “left/right, Go/No-Go” reaction time (RT) task was used. One monkey was trained to perform wrist flexion/extension movements to align a pointer with visual targets. The visual display was organized to provide a two-dimensional stimulus: side (an easy discrimination between left and right targets) which determined movement direction, and distance (a difficult discrimination between distal and proximal targets) which determined whether or not the movement was to be made. Changes in neuronal activity, when they were time-locked to the stimulus, were almost similar in the Go and No-Go trials, and when they were time-locked to movement onset, were markedly reduced in No-Go as compared to Go trials.

In Experiment 2, a stimulus-response compatibility (SRC) task was used. Two monkeys were trained to align a pointer with visual targets, on either left or right. In the spatially “compatible” trials, they had to point at the stimulus position, whereas in the “incompatible” trials, they had to point at the target located in the opposite side. For 12.5% of neurons, changes in activity associated with incompatible trials looked like changes in activity associated with movements performed in the opposite direction during compatible trials, thus suggesting the hypothesis of an automatic activation of the congruent, but incorrect response.

Results of both experiments provide evidence for a partial transmission of information from visual to motor cortical areas: that is, in the No-Go trials of the first task, information about movement direction, before the decision to perform or not this movement was made, and, in the incompatible trials of the SRC task, information about the congruent, but incorrect response, before the incongruent, but correct response was programmed.     

Intersensory redundancy educates selective attention in bobwhite quail embryos

We assessed whether exposure to amodal properties in bimodal stimulation (e.g. rhythm, rate, duration) could educate attention to amodal properties in subsequent unimodal stimulation during prenatal development. Bobwhite quail embryos were exposed to an individual bobwhite maternal call under several experimental and control conditions during the day prior to hatching. Experimental groups received redundant auditory and visual exposure to the temporal features of an individual maternal call followed by unimodal auditory exposure to the same call immediately or after a 2-hr or 4-hr delay. Control groups received (1) the same exposure but in the reverse sequence (unimodal --> redundant bimodal), (2) asynchronous bimodal --> unimodal, (3) only unimodal exposure, or (4) only bimodal exposure. All experimental groups showed a significant preference for the familiar maternal call over a novel maternal call when tested 2 days after hatching, whereas none of the control groups showed a significant preference for the familiar call. These results indicate that intersensory redundancy can direct attention to amodal properties in bimodal stimulation and educate attention to the same amodal properties in subsequent unimodal stimulation where no intersensory redundancy is available.     

The effects of kinetic occlusion and categorization on amodal completion. A comment on Gerbino and Salmaso (1987)

The experimental study of Gerbino and Salmaso (1987) focused on the process of amodal completion, i.e. the process that leads to the impression of a total pattern in the absence of information about all parts of the pattern. One of the questions that they have attempted to answer with respect to this process is whether or not it depends on the categorization of the modal part (i.e. the part about which visual information is present) as a modification of a complete prototype. The major conclusion of their experiments is that amodal completion does not depend upon categorization. Furthermore, they interpret their results as indicating that neither the truncated form of the occluded part nor the unification/segregation of contour segments are phenomenally real. We designed some variants of the tunnel effect to demonstrate (1) that the explicit categorization of some parts of the input can have an influence on amodal completion and (2) that the truncated form of a pattern can be phenomenally real. Our results corroborate these hypotheses and can, therefore, interpreted as refuting two rather central claims of Gerbino and Salmaso (1987). The conclusion of our experiment must be that amodal completion is not as primary as they have supposed but can be influenced by kinetic occlusion and categorization stored in visual memory.     

Integrating face and voice in person perception

Integration of information from face and voice plays a central role in our social interactions. It has been mostly studied in the context of audiovisual speech perception: integration of affective or identity information has received comparatively little scientific attention. Here, we review behavioural and neuroimaging studies of face-voice integration in the context of person perception. Clear evidence for interference between facial and vocal information has been observed during affect recognition or identity processing. Integration effects on cerebral activity are apparent both at the level of heteromodal cortical regions of convergence, particularly bilateral posterior superior temporal sulcus (pSTS), and at 'unimodal' levels of sensory processing. Whether the latter reflects feedback mechanisms or direct crosstalk between auditory and visual cortices is as yet unclear.     

Infants’ perception of rhythm and tempo in unimodal and multimodal stimulation: A developmental test of the intersensory redundancy hypothesis

Research has demonstrated that young infants can detect a change in the tempo and the rhythm of an event when they experience the event bimodally (audiovisually), but not when they experience it unimodally (acoustically or visually). According to Bahrick and Lickliter (2000, 2002), intersensory redundancy available in bimodal, but not in unimodal, stimulation directs attention to the amodal properties of events in early development. Later in development, as infants become more experienced perceivers, attention becomes more flexible and can be directed toward amodal properties in unimodal and bimodal stimulation. The present study tested this developmental hypothesis by assessing the ability of older, more perceptually experienced infants to discriminate the tempo or rhythm of an event, using procedures identical to those in prior studies. The results indicated that older infants can detect a change in the rhythm and the tempo of an event following both bimodal (audiovisual) and unimodal (visual) stimulation. These results provide further support for the intersensory redundancy hypothesis and are consistent with a pattern of increasing specificity in perceptual development.     

Cortical substrates of taste aversion learning: dorsal prepiriform (insular) lesions disrupt taste aversion learning

The functional relation between restricted damage to ventral primary somatosensory neocortex and the ability of rats to acquire conditioned taste aversion (CTA( was examined by a combination of behavioral and neurohistological techniques. Lesions confined exclusively to the established gustatory neocortex (GN) did not disrupt CTA acquisition, nor did lesions confined to suprarhinal cortical areas ventral to the GN. Lesions that encroached on dorsal prepiriform and insular cortices produced CTA acquisition deficits and damaged a large proportion of efferent projections to the prefrontal and precentral neocortex. In a second experiment, lesions of dorsal prepiriform and insular cortices did not modify taste preference-aversion threshold to any of the four taste modalities. It is concluded tha ventral somatosensory neocortical fields, including the established GN, do not mediate CTA acquisition and that rhinal cortices ventral and posterior to the GN are preferentially involved in associative learning for tastes and illness.     

Increasing task difficulty enhances effects of intersensory redundancy: testing a new prediction of the Intersensory Redundancy Hypothesis

Prior research has demonstrated intersensory facilitation for perception of amodal properties of events such as tempo and rhythm in early development, supporting predictions of the Intersensory Redundancy Hypothesis (IRH). Specifically, infants discriminate amodal properties in bimodal, redundant stimulation but not in unimodal, nonredundant stimulation in early development, whereas later in development infants can detect amodal properties in both redundant and nonredundant stimulation. The present study tested a new prediction of the IRH: that effects of intersensory redundancy on attention and perceptual processing are most apparent in tasks of high difficulty relative to the skills of the perceiver. We assessed whether by increasing task difficulty, older infants would revert to patterns of intersensory facilitation shown by younger infants. Results confirmed our prediction and demonstrated that in difficult tempo discrimination tasks, 5‐month‐olds perform like 3‐month‐olds, showing intersensory facilitation for tempo discrimination. In contrast, in tasks of low and moderate difficulty, 5‐month‐olds discriminate tempo changes in both redundant audiovisual and nonredundant unimodal visual stimulation. These findings indicate that intersensory facilitation is most apparent for tasks of relatively high difficulty and may therefore persist across the lifespan.     

Derived relational responding as generalized operant behavior

The major aim of the present study was to demonstrate that derived relational responding may be viewed as a form of generalized operant behavior. In Experiment 1, 4 subjects were divided into two conditions (2 in each condition). Using a two-comparison matching-to-sample procedure, all subjects were trained and tested for the formation of two combinatorially entailed relations. Subjects were trained and tested across multiple stimulus sets. Each set was composed of novel stimuli. Both Conditions 1 and 2 involved explicit performance-contingent feedback presented at the end of each block of test trials (i.e., delayed feedback). In Condition 1, feedback was accurate (consistent with the experimenter-designated relations) following exposure to the initial stimulus sets. When subjects' responding reached a predefined mastery criterion, the feedback then switched to inaccurate (not consistent with the experimenter-designated relations) until responding once again reached a predefined criterion. Condition 2 was similar to Condition 1, except that exposure to the initial stimulus sets was followed by inaccurate feedback and once the criterion was reached feedback switched to accurate. Once relational responding emerged and stabilized, response patterns on novel stimulus sets were controlled by the feedback delivered for previous stimulus sets. Experiment 2 replicated Experiment 1, except that during Conditions 3 and 4 four comparison stimuli were employed during training and testing. Experiment 3 was similar to Condition 1 of Experiment 1, except that after the mastery criterion was reached for class-consistent responding, feedback alternated from accurate to inaccurate across each successive stimulus set. Experiment 4 involved two types of feedback, one type following tests for mutual entailment and the other type following tests for combinatorial entailment. Results from this experiment demonstrated that mutual and combinatorial entailment may be controlled independently by accurate and inaccurate feedback. Overall, the data support the suggestion, made by relational frame theory, that derived relational responding is a form of generalized operant behavior.     

Simultaneous completions of modal and amodal figures: Visual evoked potentials reveal asymmetrical interference effects

Modal and amodal completion processes are thought to affect the emergence and the potency of each other. To see whether one dominates over the other, we measured the Npd, that is, the negativity of visual-evoked potentials whose amplitude increases with perceptual difficulty. In the first experiment, the Npds to illusory (modal) squares and diamonds placed over amodal diamonds and squares, respectively, were found to be greater when targets were the amodal figures than when targets were the modal figures, thus suggesting that modal figures created more interference. A second experiment showed that greater Npds were specific to interference and not to the greater difficulty at focusing on amodal figures rather than on modal figures. A third experiment showed that the interference also occurs with real figures in replacement of modal ones. Overall, the results suggest a certain dominance of modal completion over amodal completion when both occur in the same display.