Brain network interactions in auditory, visual and linguistic processing

In the paper, we discuss the importance of network interactions between brain regions in mediating performance of sensorimotor and cognitive tasks, including those associated with language processing. Functional neuroimaging, especially PET and fMRI, provide data that are obtained essentially simultaneously from much of the brain, and thus are ideal for enabling one to assess interregional functional interactions. Two ways to use these types of data to assess network interactions are presented. First, using PET, we demonstrate that anterior and posterior perisylvian language areas have stronger functional connectivity during spontaneous narrative production than during other less linguistically demanding production tasks. Second, we show how one can use large-scale neural network modeling to relate neural activity to the hemodynamically-based data generated by fMRI and PET. We review two versions of a model of object processing - one for visual and one for auditory objects. The regions comprising the models include primary and secondary sensory cortex, association cortex in the temporal lobe, and prefrontal cortex. Each model incorporates specific assumptions about how neurons in each of these areas function, and how neurons in the different areas are interconnected with each other. Each model is able to perform a delayed match-to-sample task for simple objects (simple shapes for the visual model; tonal contours for the auditory model). We find that the simulated electrical activities in each region are similar to those observed in nonhuman primates performing analogous tasks, and the absolute values of the simulated integrated synaptic activity in each brain region match human fMRI/PET data. Thus, this type of modeling provides a way to understand the neural bases for the sensorimotor and cognitive tasks of interest.     

Peripheral visual processing

An attempt was made to examine the development of the ability to identify stimuli presented to peripheral vision in several different tasks. Five- and 8-year-old children and college adults saw, for 20 msec, either a single figure at 1°, 2°, 4°, or 6° of visual angle from the fovea (singleform condition) or an off-foveal figure with an additional figure at the fovea (double-form condition). In the double-form conditions, the subjects were required to identify either the peripheral figure only (double-form presentation) or both figures (double-form report). The main effects of Age, Distance, and Form Condition were significant. Accuracy improved with increasing age and with decreasing distance. The Form Condition effect reflected lower accuracy in the two double-form conditions than in the single-form condition. Interestingly, there was no difference between the two double-form conditions, suggesting that the mere presence of a foveal stimulus, with instructions to ignore it, produces as much decrement in peripheral performance as when subjects are told to fully process and report the foveal stimulus. Also, there was no interaction between Form Condition and Distance, suggesting that the label “tunnel vision” may be misleading, since the presence of the foveal stimulus seems to have an equal effect on all peripheral locations and does not really “restrict” the size of the effective visual field.     

Right-hemisphere interactions in picture-word processing

The processing of pictures was investigated in three experiments which eliminated the response effects involved in naming. When a categorization task was used, clear advantages in response latency and accuracy were observed for left visual-field (LVF) presentations. This was in contrast to previous investigations which have used a naming task and which have reported right visual-field (RVF) advantages. When a distracting word was added to the display, the pattern of influence also changed from that reported previously. The use of naming tasks has indicated predominantly left-hemisphere effects, with demonstrations of interactions between pictures and words in the RVF. With a categorization task in Experiment 2, however, the only effective words were those related in meaning to the picture, and only when they were projected to the right hemisphere. The third experiment confirmed the LVF advantage for picture processing with masked displays, but found no reliable asymmetry with unmasked presentations. The pattern of semantic facilitation was also confirmed with the masked displays, but when the mask was removed an inhibition effect replaced the facilitation effect. These effects are interpreted as indicating that picture recognition is localized within the right cerebral hemisphere. It is suggested that the facilitating effect of related words is restricted to the left hemisphere because it is an effect upon recognition processes, whereas the inhibition effect reflects response competition. It is also suggested that previous reports of left-hemisphere interference effects are due to effects of response competition in naming tasks.     

Dependent spatial channels in visual processing

The present experiments were designed to test whether or not processing in visual information channels defined by spatial position is independent in the visual search paradigm. In Experiment 1 subjects were asked to judge whether or not a red square was present in a display of two colored geometric figures. Their mean reaction time (RT) to respond no to a “divided target” display in which one figure was red and the other was a square was about 100 msec longer than to control displays containing either two red circles or two green squares. This result is inconsistent with a spatially serial independent-channel model and with many spatially parallel independent-channel models. The relatively slow responding to divided target displays was replicated in Experiments 2 and 3, when subjects judged whether or not an “A” was present in a display of two alphanumeric characters, and a divided target display was one which contained two features of “A.” Experiments 4 and 5 demonstrated that the dependence observed in the first three experiments was probably the result of two mechanisms: crosstalk integration, whereby the target features are integrated across the two spatial channels, and repetition facilitation, whereby processing is facilitated (in some cases) when the two figures in the display are physically identical. Experiment 6 suggested that subjects organized the display in terms of spatial channels even when the task allowed subjects to ignore spatial location.     

Context dependence in visual feature processing

Summary Dependence in visual feature processing was studied with two identification experiments using briefly exposed stimuli. The basic stimuli consisted of two orthogonal line segments which formed either one of the four angles of a square, and the subject had to identify the location of the horizontal and vertical line segment making up a stimulus. In the first experiment, the two orthogonal line segments were sometimes separated by a gap. In the second experiment, either a 45° diagonal or a 135° diagonal or both were added to the right angles. With the observed response frequencies two forms of independence in feature detection were tested. Feature detection is called state independent if the detection of a feature is independent of the detection of another feature. According to context independence the detection of a feature is independent of the orientation and location of other features in the stimulus. Feature detection was shown to be context dependent whereas the hypothesis of state independence was not rejected, and state independence was unrelated to the size of the gap between the two orthogonal line segments in the first experiment. It was argued that the context effects obtained (context dependence) had occurred early in processing during the automatic activation of representational units corresponding to features before controlled search, interpretation and decision processes became involved.     

Parallel processing in visual same-different decisions

Two to 13 geometrical shapes were exposed simultaneously to S who decided whether all shapes were the same or whether one was different from the rest. Correct different decisions were usually faster than correct same decisions, but latency was independent of the number of shapes presented. We conclude that input from all the shapes was simultaneously processed into either one or two shape categories, and that a decision-theory choice was made between “same” (one shape category) and “different” (two shape categories) independent of the total number of shapes. This parallel processing is thought to be a characteristic of codable stimuli. Some observed same-different latency reversals were probably caused by a shift in the same-different criterion on the continuum for one- vs two-category decisions.     

Lateral masking in visual information processing

When unrelated letter strings are presented tachistoscopically, the end letters are reported more often than their neighbors; and when spaces are inserted into strings, performance on certain adjacent letters is superior to performance on those letters when no spaces were present. An experiment was conducted to determine the nature of those spacing effects. Letter strings were presented at a variety of retinal locations, and spaces were inserted into different positions in the instructed left-right processing order. The space effect was unrelated to processing order, but it was dependent on retinal location. To account for the various asymmetries, it was necessary to postulate that letters tend to interact with adjacent lettersand that the interaction was not spatially symmetric. Furthermore, it was found that spaces had greater effects on “right-hand” letters than on symmetric letters. It was therefore concluded that letters interact at the feature level.     

Hemispheric differences in processing visual patterns

The dichotomies verbal/visuospatial, serial/parallel and analytic/holistic are reviewed with respect to differences in hemispheric processing. A number of experimental parameters may be varied in such tasks, and together with certain frequently-occurring weaknesses of experimental design may account for the often discrepant results hitherto reported. The above factors are systematically reviewed, and three further experiments are reported which attempt to fill in the missing designs. Further evidence is given in support of the hypothesis that right-hemisphere superiority is most apparent in processes leading to identity matching. It is quantitative rather than qualitative, and may depend upon operations on the entire gestalt, such as holistic matching, mental rotation, reflection, distortion, etc., rather than, e.g., simultaneous (parallel) processing of discretely analysed or isolated features or elements. On the other hand left-hemisphere involvement in visuospatial processing is thought to reflect analysis of the configuration into its separable components; such processing may be either serial or parallel, and may frequently lead to a judgement different.     

Orthographic processing in visual word identification

A series of experiments is reported examining orthographic priming effects between briefly presented pairs of letter strings. The experiments investigate the effects of the number and position of letters shared by primes and targets, and the effects of prime-target length. Priming effects increase nonlinearly as a function of both the number and the position of shared letters, and they are dependent on the positions of letters relative to both the end positions in the string and to the identities of their nearest neighbours. There is little effect of absolute string length on priming. These priming effects can be distinguished from intrusion errors where letters from primes are reported in response to targets. An account of orthographic processing is outlined which attributes priming to cooperative interactions between coarse relative-position coded letter cluster representations activated by primes and targets. The implications of the findings for understanding other effects in word recognition and reading are discussed.     

Configurational effects in visual information processing

These experiments show that the perceptual organization of a multielement display affects both the speed and accuracy with which a target letter in it is detected. The first two experiments show that a target is detected more poorly if it is arranged in good form (a perceptual Gestalt) with noise elements than if it is not. This effect is not confounded with target-noise proximity or display size, and it holds for stimuli terminated by the subject’s response as well as for stimuli of very brief duration. Increasing the number of noise elements can actually improve performance if the added noise elements increase the degree to which the noise elements form perceptual groups separately from the target. A third experiment tries out a new method for scaling the perceptual structure of an array, and it shows that the main features of the first two experiments can be predicted from the scaled perceptual structure of the arrays they used.     

Attentional holism in visual word processing

Summary Decisions about the identity of an entire word are made more quickly than decisions about the identity of a letter within a word. Explanations of this whole-word advantage based on the time-course of the activation of different levels of unit detectors have given way to explanations based on how attention is allocated to the output of the detectors. Three studies were carried out to examine the role of attention in the whole-word advantage. In Experiment 1, cues as to the level of decision (whole-word vs. first-letter) facilitated component processing, a finding that suggests word level information is normally the focus of attention. In Experiment 2, identification of a probe item to the right of a display was longer when subjects prepared for a first-letter rather than a whole-word decision. That is, the spatial extent of attention is wider for whole-word decisions. In Experiment 3, probe latencies were longer when subjects prepared for a whole-word decision than when they prepared for a signalled probe trial. Preparation for a whole-word decision is not automatic in the sense of being free of capacity demands. The overall pattern of results leads to the conclusion that the whole-word advantage is an instance of attentional holism.     

Capacity limitations in visual word processing

The ability of subjects to process English words in a spatially parallel manner was examined in several redundant-target detection tasks. When redundant targets were identical in a given display, processing limitations were evident in a task that required subjects to make semantic categorizations of words. However, parallel processing of identical redundant target words was exhibited in a lexical decision task that required a structural analysis of letter strings, but not an analysis of word meaning. The difference in performance in the two tasks suggests that the capacity for semantic processing is limited. Analyses designed to examine whether the redundancy gain in Experiment 2 could be attributed to limited capacity processing in conjunction with positional preferences provided evidence against this possibility. In addition, these analyses suggested that the processing times for the redundant targets in Experiment 2 might be positively correlated. In the third and fourth experiments, the redundant-target displays contained two different words. Processing interference, in the form of a redundancy loss, was evident in the lexical decision task, but not in the semantic categorization task, confirming a difference in the mode of processing between the two tasks. The results provide evidence against the unlimited-capacity, parallel processing hypothesis of late selection theories of attention.     

Parallel processing in visual search asymmetry

The difficulty of visual search may depend on assignment of the same visual elements as targets and distractors-search asymmetry. Easy C-in-O searches and difficult O-in-C searches are often associated with parallel and serial search, respectively. Here, the time course of visual search was measured for both tasks with speed-accuracy methods. The time courses of the 2 tasks were similar and independent of display size. New probabilistic parallel and serial search models and sophisticated-guessing variants made predictions about time course and accuracy of visual search. The probabilistic parallel model provided an excellent account of the data, but the serial model did not. Asymptotic search accuracies and display size effects were consistent with a signal-detection analysis, with lower variance encoding of Cs than Os. In the absence of eye movements, asymmetric visual search, long considered an example of serial deployment of covert attention, is qualitatively and quantitatively consistent with parallel search processes.     

Parallel processing in complex visual discrimination

In a “same-different” reaction time (RT) task pairs of stimuli varying along one or more dimensions are presented and S is required to indicate, as rapidly as possible, whether the stimuli are physically identical or different. This task was employed in three experiments investigating the processes by which multidimensional stimuli are discriminated. The results indicated that stimulus dimensions are compared in parallel; that the time required to interrogate a dimension varies randomly across trials and is dependent upon the time required to interrogate other dimensions present in test stimuli, and that comparisons terminate upon the detection of information sufficient for a correct response.     

Dimensional interactions in vibrotactile information processing

The effect of irrelevant dimensional variation on the processing of vibrotactile stimuli was measured. Six observers performed a speeded classification task with stimuli varying along the dimensions of pitch and loudness. Choice reaction times were obtained for stimuli differing on one dimension alone, on two correlated dimensions, or on two orthogonally varied dimensions. Compared to one-dimension performance, reaction times were faster in the correlated condition and slower in the orthogonal condition. In general, these findings agreed with similar experiments in other modalities, with the exception that the effects in this study tended to be stronger for cases in which loudness, rather than pitch, was the relevant dimension. The results are explained in terms of the integrality of pitch and loudness and of the relative discriminability of dimensions.     

Adaptive processing of visual motion

Three studies relating perception of motion to stimulus uncertainty are reported. Generally, detectability declines when the observer is uncertain about the direction in which a target will move, but the visibility loss associated with direction uncertainty can be attenuated if the observer has adequate practice. This attenuation seems to depend upon the observer's ability to switch among directionally selective visual mechanisms in an adaptive fashion. The implications of these findings for models of motion detection are discussed.     

Depth of visual information processing

The depth of visual information processing is identified with the longest randomly-generated binary-encoded spatial pattern whose partial repetitions can be detected. In contrast with the auditory detection of sequentially-presented constraints, the depth of perceptible visual encoding of spatially-presented constraints is sharply limited. Depths of about 35 were achieved for one-dimensional constraints; depths of perhaps 3 or 4 were achieved for two-dimensional constraints imposed in only one direction and depths of perhaps 2 for two-dimensional constraints imposed in two directions. With one-dimensional constraints, it is shown that the inferred depth of processing is partially determined by the number of pattern representations and by the spatial distance between successive representation for visual displays of fixed size.     

Learned predictiveness speeds visual processing

When humans learn that the presence of a cue predicts the likelihood of an outcome, they can exploit this learned predictiveness, such that formation of subsequent associations between that cue and new outcomes is facilitated. Could such enhanced selection for association arise early enough to facilitate low-level visual processing? In a test of this possibility, adult volunteers first engaged in a value-learning task involving faces that were differentially predictive of monetary wins or losses. Later, in a simple recognition task, these faces were briefly presented for a variable duration and then masked. The critical presentation duration needed to produce criterion-level recognition was measured to index the visual processing speed for each learned face. Critical duration was significantly shorter for stimuli with high learned predictiveness than for stimuli with low learned predictiveness, regardless of whether they were associated with wins or losses. These results show that neural mechanisms involved in predicting future outcomes are able to modulate visual processing efficiency, probably via cortical feedback processes.     

Emotional distraction unbalances visual processing

Brain mechanisms used to control nonemotional aspects of cognition may be distinct from those regulating responses to emotional stimuli, with activity of the latter being detrimental to the former. Previous studies have shown that suppression of irrelevant emotional stimuli produces a largely right-lateralized pattern of frontal brain activation, thus predicting that emotional stimuli may invoke temporary, lateralized costs to performance on nonemotional cognitive tasks. To test this, we briefly (85 ms) presented a central, irrelevant, expressive (angry, happy, sad, or fearful) or neutral face 100 ms prior to a letter search task. The presentation of emotional versus neutral faces slowed subsequent search for targets appearing in the left, but not the right, hemifield, supporting the notion of a right-lateralized, emotional response mechanism that competes for control with nonemotional cognitive processes. Presentation of neutral, scrambled, or inverted neutral faces produced no such laterality effects on visual search response times.