Are depictive gestures like pictures? commonalities and differences in semantic processing

Conversation is multi-modal, involving both talk and gesture. Does understanding depictive gestures engage processes similar to those recruited in the comprehension of drawings or photographs? Event-related brain potentials (ERPs) were recorded from neurotypical adults as they viewed spontaneously produced depictive gestures preceded by congruent and incongruent contexts. Gestures were presented either dynamically in short, soundless video-clips, or statically as freeze frames extracted from gesture videos. In a separate ERP experiment, the same participants viewed related or unrelated pairs of photographs depicting common real-world objects. Both object photos and gesture stimuli elicited less negative ERPs from 400 to 600 ms post-stimulus when preceded by matching versus mismatching contexts (dN450). Object photos and static gesture stills also elicited less negative ERPs between 300 and 400 ms post-stimulus (dN300). Findings demonstrate commonalities between the conceptual integration processes underlying the interpretation of iconic gestures and other types of image-based representations of the visual world.     

Early and late mismatch negativity elicited by words and speech-like stimuli in children

In auditory perception the brain's attentional and preattentional mechanisms select certain stimuli for preferential processing and filter out irrelevant input. This study investigated nonattentive auditory processing in children. Event-related potentials (ERPs) provide a means to study neural correlates related to language and speech-sound processing. Mismatch negativity (MMN) is an ERP wave that indicates attention-independent perceptual change detection. In this study cortical ERPs were elicited by complex tones, naturally spoken words, and pseudowords, with each stimulus type containing equal acoustical elements. Tones elicited a bifurcated mismatch negativity (MMN), with early MMN (peaking at 150-200 ms) being more dominant. On the other hand, words elicited a strong late MMN, peaking at about 400-450 ms after stimulus onset. The MMN wave form was significantly weaker for pseudowords than for words. The late MMN wave, especially for word differences, was found to reflect summating MMN generators and memory trace formation on gestalt bases. Results suggest that the auditory processing, even nonattended, is highly associated with the cognitive meaning of the stimuli.     

Dissociation of event-related potentials indexing arousal and semantic cohesion during emotional word encoding

Event-related potential (ERP) studies have shown that emotional stimuli elicit greater amplitude late positive-polarity potentials (LPPs) than neutral stimuli. This effect has been attributed to arousal, but emotional stimuli are also more semantically coherent than uncategorized neutral stimuli. ERPs were recorded during encoding of positive, negative, uncategorized neutral, and categorized neutral words. Differences in LPP amplitude elicited by emotional versus uncategorized neutral stimuli were evident from 450 to 1000 ms. From 450 to 700 ms, LPP effects at midline and right hemisphere frontal electrodes indexed arousal, whereas LPP effects at left hemisphere centro-parietal electrodes indexed semantic cohesion. This dissociation helps specify the processes underlying emotional stimulus encoding, and suggests the need to control for semantic cohesion in emotional information processing studies.     

Multisensory processing in the redundant-target effect: a behavioral and event-related potential study

Participants respond more quickly to two simultaneously presented target stimuli of two different modalities (redundant targets) than would be predicted from their reaction times to the unimodal targets. To examine the neural correlates of this redundant-target effect, event-related potentials (ERPs) were recorded to auditory, visual, and bimodal standard and target stimuli presented at two locations (left and right of central fixation). Bimodal stimuli were combinations of two standards, two targets, or a standard and a target, presented either from the same or from different locations. Responses generally were faster for bimodal stimuli than for unimodal stimuli and were faster for spatially congruent than for spatially incongruent bimodal events. ERPs to spatially congruent and spatially incongruent bimodal stimuli started to differ over the parietal cortex as early as 160 msec after stimulus onset. The present study suggests that hearing and seeing interact at sensory-processing stages by matching spatial information across modalities.     

Attenuated audiovisual integration in middle-aged adults in a discrimination task

Numerous studies have focused on the diversity of audiovisual integration between younger and older adults. However, consecutive trends in audiovisual integration throughout life are still unclear. In the present study, to clarify audiovisual integration characteristics in middle-aged adults, we instructed younger and middle-aged adults to conduct an auditory/visual stimuli discrimination experiment. Randomized streams of unimodal auditory (A), unimodal visual (V) or audiovisual stimuli were presented on the left or right hemispace of the central fixation point, and subjects were instructed to respond to the target stimuli rapidly and accurately. Our results demonstrated that the responses of middle-aged adults to all unimodal and bimodal stimuli were significantly slower than those of younger adults (p < 0.05). Audiovisual integration was markedly delayed (onset time 360 ms) and weaker (peak 3.97%) in middle-aged adults than in younger adults (onset time 260 ms, peak 11.86%). The results suggested that audiovisual integration was attenuated in middle-aged adults and further confirmed age-related decline in information processing.     

记忆机制中再认的事件相关电位

ERP对于记忆的研究有独到之处,在研究方法是有外显的直接范式和内隐的间接范式,分别表现为ERP新旧效应和重复效应,即在刺激200ms以后的正向偏移.对于记忆提取的ERP效应已有一些理论解释,但在听觉记忆、汉字记忆以及记忆的脑内机制等方面还需要进一步的深入研究.     

Auditory awareness negativity is an electrophysiological correlate of awareness in an auditory threshold task

One theory of visual awareness proposes that electrophysiological activity related to awareness occurs in primary visual areas approximately 200 ms after stimulus onset (visual awareness negativity: VAN) and in fronto-parietal areas about 300 ms after stimulus onset (late positivity: LP). Although similar processes might be involved in auditory awareness, only sparse evidence exists for this idea. In the present study, we recorded electrophysiological activity while subjects listened to tones that were presented at their own awareness threshold. The difference in electrophysiological activity elicited by tones that subjects reported being aware of versus unaware of showed an early negativity about 200 ms and a late positivity about 300 ms after stimulus onset. These results closely match those found in vision and provide convincing evidence for an early negativity (auditory awareness negativity: AAN), as well as an LP. These findings suggest that theories of visual awareness are also applicable to auditory awareness.     

Vibrotactile frequency discrimination at short durations

This experiment investigated how frequency discrimination of a sinusoidal, mechanical vibration applied to the tip of the right index finger is affected by shortening the duration of the stimuli from 200 ms to 30 ms. Using a standard stimulus of 100 Hz at 30 dB above threshold, seven comparison frequencies (at intervals of 10 Hz) were judged as higher or lower in frequency according to the method of constant differences. Vibrotactile stimuli were matched for subjective intensity across both frequency and duration. Difference limens for vibrotactile frequency were found to decline slightly from 200 ms to 50 ms (attributable to practice) and to increase noticeably at 30 ms. This result is discussed in relation to the seemingly contradictory results for auditory pitch discrimination.     

Ideomotor compatibility in the psychological refractory period effect: 29 years of oversimplification

Four experiments examined whether the psychological refractory period (PRP) effect can be eliminated with ideomotor compatible (IM) but not stimulus-response compatible (SR) tasks, as reported by A. G. Greenwald and H. G. Shulman (1973). Their tasks were used: a left or right movement to a left- or right-pointing arrow (IM) or to the word left or right (SR) for Task 1; saying "A" or "B" (IM) or "1" or "2" (SR) to an auditory A or B for Task 2. The stimulus onset asynchronies were 0, 100, 200, 300, 500, and 1,000 ms in Experiment 1, and only 0, 100, 200, and 1,000 ms in Experiments 2-4. The arrow was in the center of the screen in Experiments 1-3 and to the left or right in Experiment 4. As in Greenwald and Shulman's Experiment 2, the instructions stated that most often the 2 stimuli would be presented simultaneously. A PRP effect was obtained in all conditions, most likely because response-selection decisions are required even for IM tasks.     

内隐与外显记忆编码阶段脑机制的重叠与分离

记忆包括编码和提取两个重要阶段, 以往研究多关注内隐和外显记忆在提取阶段的脑机制分离, 那二者在编码阶段的脑机制是否不同？为探讨这个问题, 该实验采用两阶段的迫选再认方式, 把学习项目区分成三类：随后被选择且告知为见过的项目(记住), 随后被选择但告知为猜测的项目 (启动)以及随后未被选择的项目(忘记)。结果表明, 从200 ms开始的颞区负走向Dm效应为内隐记忆(启动vs忘记)所特有, 反映了对刺激的知觉加工过程, 400~500 ms前额区正走向的Dm效应为外显记忆(记住vs忘记)所特有, 反映了对刺激的精细加工过程, 而200~300 ms中央区及600 ms开始的顶区负走向的Dm效应为两种记忆类型所共有, 它们分别反映了对刺激的注意状态以及把编码后的刺激信息登记进相应记忆系统的过程。因此, 内隐记忆与外显记忆在编码阶段的脑机制既存在着分离, 也存在着重叠的现象。     

Feature processing during high-rate auditory selective attention

Auditory event-related brain potentials (ERPs) and reaction times were, analyzed in a selective attention task in which subjects attended to tone pips presented at high rates-Xinterstimulua intervals [ISIs] of 40-200 msec). Subjects responded to infrequent target tones of a specified frequency (250 or 4000 Hz) and location (left or right ear) that were louder than otherwise identical tones presented randomly to the left and right ears. Negative difference (Nd) waves were isolated by subtracting ERPs to tones with no target features from ERPs to the same tones when they shared target location, frequency, or both frequency and location cues. Nd waves began 60-70 msec after tone onset and lasted until 252–350 msec after tone onset, even for tones with single attended cues. The duration of Nd waves exceeded the ISIs between successive tones, implying that several stimuli underwent concurrent analysis. Nd waves associated with frequency processing had scalp distributions different from those associated with location processing, implying that the features were analyzed in distinct cortical areas. Nd waves specific to auditory feature conjunction were isolated. These began at latencies of 110–120 msec, some 30-40 msec after the Nds to single features. The relative timing of the different Nd waves suggests that auditory feaure conjunction begins after a brief parallel analysis of individual features but before feature analysis is complete.     

Electrophysiological correlates of rapid auditory and linguistic processing in adolescents with specific language impairment

Brief tonal stimuli and spoken sentences were utilized to examine whether adolescents (aged 14;3-18;1) with specific language impairments (SLI) exhibit atypical neural activity for rapid auditory processing of non-linguistic stimuli and linguistic processing of verb-agreement and semantic constraints. Further, we examined whether the behavioral and electrophysiological indices for rapid auditory processing were correlated with those for linguistic processing. Fifteen adolescents with SLI and 15 adolescents with normal language met strict criteria for displaying consistent diagnoses from kindergarten through the eighth grade. The findings provide evidence that auditory processing for non-linguistic stimuli is atypical in a significant number of adolescents with SLI compared to peers with normal language and indicate that reduced efficiency in auditory processing in SLI is more vulnerable to rapid rates (200ms ISI) of stimuli presentation (indexed by reduced accuracy, a tendency for longer RTs, reduced N100 over right anterior sites, and reduced amplitude P300). Many adolescents with SLI displayed reduced behavioral accuracy for detecting verb-agreement violations and semantic anomalies, along with less robust P600s elicited by verb-agreement violations. The results indicate that ERPs elicited by morphosyntactic aspects of language processing are atypical in many adolescents with SLI. Additionally, correlational analyses between behavioral and electrophysiological indices of processing non-linguistic stimuli and verb-agreement violations suggest that the integrity of neural functions for auditory processing may only account for a small proportion of the variance in morphosyntactic processing in some adolescents.     

记忆编码中目标与背景的情绪一致性对记忆提取的影响

摘 要 本研究将经典情绪启动范式和学习-再认范式相结合,探讨学习时启动与目标的情绪一致性对目标再认的影响。在记忆编码阶段,以情绪面孔图片为启动刺激、一般情绪图片为目标刺激,要求被试对目标图片的效价进行判断;在提取阶段向被试呈现一般情绪图片,要求被试判断是否在前一阶段见过。结果表明,编码阶段的加工背景对记忆提取产生了显著影响。一致启动促进了认知加工与记忆编码,提高了再认正确率。ERP结果也表明,在目标刺激呈现后300-500ms,编码时的情绪一致性对再认阶段消极目标诱发的ERP产生了显著影响。     

语义整合的早期加工:来自ERPs的证据

本研究借助ERPs技术,通过操纵关键词与当前语境的语义合理性关系,考察在严格限定关键词词频变异性的情况下,语义整合过程能否反映在P2效应上。ERPs结果发现关键词呈现后,语义合理相比违背条件在右侧脑区诱发了更大的P2成分。该结果支持如果关键词低水平的变量信息(词频等)得到适当控制,ERPs技术可以检测到发生在250ms以前的语义整合过程。     

主动性控制在语言转换线索加工阶段的作用

利用ERP技术,考察个体的主动性控制在语言转换线索加工阶段的作用。实验采用线索化的语言转换范式,要求汉–英双语者根据线索使用汉语或英语对图片进行命名,记录线索加工阶段的脑电成分。采用AX连续执行测验（AX-Continuous Performance Test）中AY试次的错误率作为主动性控制参与程度的指标。结果发现,在线索呈现后的200-350 ms和350-500 ms两个时间窗口,转换试次比重复试次诱发了更负的N2波幅和晚期负波,而且AY试次的错误率可以显著地预测转换试次的N2和晚期负波的波幅。结果表明,语言转换的线索加工可区分为由主动性控制调节的两个阶段,即“与非目标语言任务图式的分离”和“转入目标语言任务图式”的阶段。本研究为主动性控制参与语言转换过程提供了直接的实验证据。     

Cross-modal training induces changes in spatial representations early in the auditory processing pathway

In the ventriloquism aftereffect, brief exposure to a consistent spatial disparity between auditory and visual stimuli leads to a subsequent shift in subjective sound localization toward the positions of the visual stimuli. Such rapid adaptive changes probably play an important role in maintaining the coherence of spatial representations across the various sensory systems. In the research reported here, we used event-related potentials (ERPs) to identify the stage in the auditory processing stream that is modulated by audiovisual discrepancy training. Both before and after exposure to synchronous audiovisual stimuli that had a constant spatial disparity of 15°, participants reported the perceived location of brief auditory stimuli that were presented from central and lateral locations. In conjunction with a sound localization shift in the direction of the visual stimuli (the behavioral ventriloquism aftereffect), auditory ERPs as early as 100 ms poststimulus (N100) were systematically modulated by the disparity training. These results suggest that cross-modal learning was mediated by a relatively early stage in the auditory cortical processing stream.     

The gradient of spatial auditory attention in free field: An event-related potential study

Young adult subjects attended selectively to brief noise bursts delivered in free field via a horizontal array of seven loudspeakers spaced apart by 9° of angle. Frequent “standard” stimuli (90%) and infrequent “target/deviant” stimuli (10%) of increased bandwidth were delivered at a fast rate in a random sequence equiprobably from each speaker. In separate runs, the subjects’ task was to selectively attend to the leftmost, center, or rightmost speaker and to press a button to the infrequent “target” stimuli occurring at the designated spatial location. Behavioral detection rates and concurrently recorded event-related potentials (ERPs) indicated that auditory attention was deployed as a finely tuned gradient around the attended sound source, thus providing support for gradient models of auditory spatial attention. Furthermore, the ERP data suggested that the spatial focusing of attention was achieved in two distinct stages, with an early more broadly tuned filtering of inputs occurring over the first 80–200 msec after stimulus onset, followed by a more narrowly focused selection of attended-location deviants that began at around 250 msec and closely resembled the behavioral gradient of target detections.     

Contextual modulation of N400 amplitude to lexically ambiguous words

Though much is known about the N400 component, an event-related EEG potential that is sensitive to semantic manipulations, it is unclear whether modulations of N400 amplitude reflect automatic processing, controlled processing, or both. We examined this issue using a semantic judgment task that manipulated local and global contextual cues. Word triplets (prime-noun-target, e.g., finance-bank-money) were sequentially presented on a computer screen (500 ms duration, 1000 ms stimulus onset asynchrony), in which the second word was a homograph. The first word (prime) created a neutral-, dominant-meaning- or subordinate-meaning-biased "global context," and the third word (target) created a dominant- or subordinate-biased "local context" that was either congruent or incongruent with the "global context" established by the first prime word. Participants were instructed to read all three words but to decide only whether the second and third words were semantically related. Event-related potentials (ERPs), specifically the N400, were recorded to the third terminal word. N400 amplitudes evoked by dominant meaning-related third words incongruent with the globally biased subordinate context (e.g., river-bridge-money) were significantly more negative than dominant endings in neutral contexts (e.g., taxi-bank-money), but not different from unrelated filler triplets. In addition, there was some indication that left hemisphere, temporal-parietal electrode sites were associated with greater N400 negativity for dominant targets in conflicting subordinate global contexts than homologous right hemisphere electrode sites, the latter of which showed significant activation to subordinate meanings in cooperating contexts. Thus, N400 amplitude was more affected by global than local context suggesting that controlled processes may take priority over automatic processes in modulating N400 amplitude, especially for left hemisphere electrode sites.     

Atypical long-latency auditory event-related potentials in a subset of children with specific language impairment

It has been proposed that specific language impairment (SLI) is the consequence of low-level abnormalities in auditory perception. However, studies of long-latency auditory ERPs in children with SLI have generated inconsistent findings. A possible reason for this inconsistency is the heterogeneity of SLI. The intraclass correlation (ICC) has been proposed as a useful statistic for evaluating heterogeneity because it allows one to compare an individual's auditory ERP with the grand average waveform from a typically developing reference group. We used this method to reanalyse auditory ERPs from a sample previously described by Uwer, Albrecht and von Suchodoletz (2002). In a subset of children with receptive SLI, there was less correspondence (i.e. lower ICC) with the normative waveform (based on the control grand average) than for typically developing children. This poorer correspondence was seen in responses to both tone and speech stimuli for the period 100-228 ms post stimulus onset. The effect was lateralized and seen at right- but not left-sided electrodes.     

Gapping: Electrophysiological evidence for immediate processing of "missing" verbs in sentence comprehension

In the present study we use event related potentials (ERPs) to explore the time course of identification and resolution of verb gaps. ERPs were recorded while participants read sentences that contained a verb gap like Ron took/sanded the planks, and Bill ? the hammer... Plausibility of the critical words (hammer) that followed the verb gap was manipulated. Relative to the plausible control (preceded by took), ERPs to the critical word in the implausible condition (preceded by sanded) showed an N400, followed by a positivity (P600). ERPs to determiners following gapped verbs showed a negativity between 100 and 300 ms, and a positivity between 300 and 500 ms compared to determiners in non-gapping constructions. These results suggest that the sentence processor recognizes a verb gap and reconstructs the verb information at the earliest possible occasion, and that this reconstruction process is different from the reconstruction of antecedents in other filler-gap constructions (e.g., WH gaps).