Brain-computer interface research comes of age: traditional assumptions meet emerging realities

Brain-computer interfaces (BCIs) could provide important new communication and control options for people with severe motor disabilities. Most BCI research to date has been based on 4 assumptions that: (a) intended actions are fully represented in the cerebral cortex; (b) neuronal action potentials can provide the best picture of an intended action; (c) the best BCI is one that records action potentials and decodes them; and (d) ongoing mutual adaptation by the BCI user and the BCI system is not very important. In reality, none of these assumptions is presently defensible. Intended actions are the products of many areas, from the cortex to the spinal cord, and the contributions of each area change continually as the CNS adapts to optimize performance. BCIs must track and guide these adaptations if they are to achieve and maintain good performance. Furthermore, it is not yet clear which category of brain signals will prove most effective for BCI applications. In human studies to date, low-resolution electroencephalography-based BCIs perform as well as high-resolution cortical neuron-based BCIs. In sum, BCIs allow their users to develop new skills in which the users control brain signals rather than muscles. Thus, the central task of BCI research is to determine which brain signals users can best control, to maximize that control, and to translate it accurately and reliably into actions that accomplish the users' intentions.     

Measuring concept semantic relatedness through common spatial pattern feature extraction on EEG signals

We study the semantic relationship between pairs of nouns of concrete objects such as “HORSE - SHEEP” and “SWING - MELON” and how this relationship activity is reflected in EEG signals. We collected 18 sets of EEG records; each set containing 150 events of stimulation. In this work we focus on feature extraction algorithms. Particularly, we highlight Common Spatial Pattern (CSP) as a method of feature extraction. Based on these latter, different classifiers were trained in order to associate a set of signals to a previously learned human answer, pertaining to two classes: semantically related, or not semantically related. The results of classification accuracy were evaluated comparing with other four methods of feature extraction, and using classification algorithms from five different families. In all cases, classification accuracy was benefited from using CSP instead of FDTW, LPC, PCA or ICA for feature extraction. Particularly with the combination CSP-Naïve Bayes we obtained the best average precision of 84.63%.     

Changing the P300 brain computer interface.

Brain-computer interfaces (BCIs) are now feasible for use as an alternative control option for those with severe motor impairments. The P300 component of the evoked potential has proven useful as a control signal. Individuals do not need to be trained to produce the signal, and it is fairly stable and has a large evoked potential. Even with recent signal classification advances, on-line experiments with P300-based BCIs remain far from perfect. We present two potential methods for improving control accuracy. Experimental results in an evoked potential BCI, used to control items in a virtual apartment, show a reduced response exists when items are accidentally controlled. The presence of a P300-like signal in response to goal items means that it can be used for automatic error correction. Preliminary results from an interface experiment using three different button configurations for a yes/no BCI task show that the configuration of buttons may affect on-line signal classification. These results will be discussed in light of the special considerations needed when working with an amyotrophic lateral sclerosis (ALS) patient.     

眼动与脑电同步记录技术在心理语言学研究中的新进展

传统的心理语言学研究手段存在一定的弊端,眼动和脑电同步记录技术具有高时间和高空间分辨率的特性,能够在自然阅读情境下探讨人在阅读中认知加工的时间进程及其相应的神经机制。本文在阐述眼动与脑电同步记录的必要性和方法的基础上,较为全面地介绍了近10年来同步记录技术在心理语言学研究中的新进展,并对该技术的应用前景进行了展望,以期对国内从事心理语言学领域的研究者有一定的启示。     

Emotion perception and recognition: An exploration of cultural differences and similarities

The electroencephalogram (EEG) is a powerful method for investigation of different cognitive processes. Recently, EEG analysis became very popular and important, with classification of these signals standing out as one of the mostly used methodologies. Emotion recognition is one of the most challenging tasks in EEG analysis since not much is known about the representation of different emotions in EEG signals. In addition, inducing of desired emotion is by itself difficult, since various individuals react differently to external stimuli (audio, video, etc.). In this article, we explore the task of emotion recognition from EEG signals using distance-based time-series classification techniques, involving different individuals exposed to audio stimuli. Furthermore, since some of the participants in the experiment do not understand the language of the stimuli, we also investigate the impact of language understanding on emotion perception. Using time-series distances as features for the construction of new data representations, applied here for the first time to emotion recognition and related tasks, lead to excellent classification performance, indicating that differences between EEG signals can be used to build successful models for recognition of emotions, individuals, and other related tasks. In the process, we observed that cultural differences between the subjects did not have a significant impact on the recognition tasks and models.     

Independent component analysis: an introduction

Independent component analysis (ICA) is a method for automatically identifying the underlying factors in a given data set. This rapidly evolving technique is currently finding applications in analysis of biomedical signals (e.g. ERP, EEG, fMRI, optical imaging), and in models of visual receptive fields and separation of speech signals. This article illustrates these applications, and provides an informal introduction to ICA.     

Exploring the effects of EEG signals on collision cases happening in the process of young drivers’ braking

Detecting mental states in drivers offers an opportunity to reduce accidents by triggering alerts and signaling the need for rest or renewed focus. Here we used electroencephalography (EEG) to measure brain signals in young drivers operating a driving simulator to detect mental states and predict accidents. We measured reaction times to unexpected hazardous events and correlated them with EEG signals measured from the frontal, parietal, and temporal cortices as well as the central sulcus (corresponding to motor cortex). We found that EEG signals in the relative beta (power in beta (13–30 Hz) relative to total power of the EEG (0.5–30 Hz)), alpha/delta, alpha/theta, beta/delta, beta/theta frequency bands were higher for collisions than successful collision avoidance, and that the key decision-making period is the 2nd second before braking. Importantly, a decision tree classifier trained on these neural signals predicted collision avoidance outcomes. Then based on random forest model, we extracted three critical neural signals (beta/delta_frontal, relative beta_parietal and relative beta_central Sulcus) to classify collision avoidance outcomes. Our findings suggest measuring EEG during driving may provide useful signals for enhancing driver safety.     

Summarization of videos by analyzing affective state of the user through crowdsource

Video summarization is one of the key techniques to access and manage a huge chunk of videos. Video summarization is used to extract the effective contents of a video sequence to generate a concise representation of its content. The involvement of crowdsourcing models in recent years is effectively used by researchers in E-commerce domain to increase the quality of the data contents. In this paper, we present a video summarization framework based on users emotion while they watch videos by analyzing cerebral activities through Electroencephalogram (EEG) signals. Three emotions, namely happy, sad and neutral have been extracted from the EEG signals. Video frames have been synchronized with EEG signals and tagged with various emotions. Finally, a crowdsourcing model has been used for effective summarization of the videos. The qualitative assessment of video summarization has been conducted with the help of user ratings using online Google Forms application. EEG signals of 28 users have been recorded while video streams of different emotions. An average accuracy of 83.93% has been recorded in emotion classification using crowdsourcing. Output summarized videos include dynamic video skims and the corresponding audio stream for better understanding.     

Changes of brain potentials in response to smoking-induced stimuli in smokers.

Changes in P300 amplitude were used as an indicator of reactivity to smoking-related stimuli in smokers. The amplitude of P300--a component of event-related brain potentials (ERPs) elicited by 10 smoking-related (craving), 10 antismoking (aversive) and 10 neutral stimuli-- was recorded in smokers (n=10) and nonsmokers (n=10). Electroencephalography (EEG) data were obtained by the Laxtha EEG-monitoring device in the EEG recording room, and were recorded at F3, F4, C3, and C4. Three-way repeated-measures analysis of variance (ANOVA) was computed on the P300 amplitudes. The factors were group (smokers, nonsmokers), stimulus (craving, aversive, neutral), and electrode location (F3, F4, C3, and C4). The main effects of stimulus were significant, but the group effects did not show significant interactions with other factors. An interesting observation was the similarity between P300 waveforms for craving and aversive stimuli in smokers. These findings could indicate that the antismoking-related response is similar to the smoking-related one.     

不同程度吸烟者内隐态度测量范式下的ERP研究

采用内隐联想测验的WIAT范式研究不同程度吸烟者的内隐态度,并用ERP技术探讨吸烟者相容与不相容任务的脑内时程变化。WIAT结果发现,只有程度深组对吸烟持明显消极态度。ERP结果表明,在不相容任务中程度深组诱发了比程度浅组更大的P2,说明程度深组对与积极相联的吸烟词早期注意增强。同时程度深组的不相容任务诱发了比相容任务更大的LPC,提示程度深组晚期对吸烟词有正性情绪加工。     

前额叶神经振荡活动反映了稳固的刺激冲突和反应冲突

为考察典型Flanker任务中练习对刺激冲突和反应冲突的影响,本研究采用练习范式记录被试完成2:1字母Flanker任务时的行为和脑电数据。行为结果发现,练习对这两类冲突无显著影响。神经震荡能量结果发现,前额叶theta能量随练习而显著减弱,这暗示刺激-反应之间的联结因练习而提升;前额叶alpha能量差异不随练习而变化,这可能反映了Flanker任务中稳定的刺激冲突和反应冲突加工机制。因此,本研究揭示了典型Flanker任务中刺激冲突和反应冲突的稳定特征。     

Non-invasive estimation of local field potentials for neuroprosthesis control

Recent experiments have shown the possibility of using the brain electrical activity to directly control the movement of robots or prosthetic devices in real time. Such neuroprostheses can be invasive or non-invasive, depending on how the brain signals are recorded. In principle, invasive approaches will provide a more natural and flexible control of neuroprostheses, but their use in humans is debatable given the inherent medical risks. Non-invasive approaches mainly use scalp electroencephalogram (EEG) signals and their main disadvantage is that these signals represent the noisy spatiotemporal overlapping of activity arising from very diverse brain regions, i.e., a single scalp electrode picks up and mixes the temporal activity of myriads of neurons at very different brain areas. In order to combine the benefits of both approaches, we propose to rely on the non-invasive estimation of local field potentials (LFP) in the whole human brain from the scalp measured EEG data using a recently developed inverse solution (ELECTRA) to the EEG inverse problem. The goal of a linear inverse procedure is to de-convolve or un-mix the scalp signals attributing to each brain area its own temporal activity. To illustrate the advantage of this approach we compare, using an identical set of spectral features, classification of rapid voluntary finger self-tapping with left and right hands based on scalp EEG and non-invasively estimated LFP on two subjects using a different number of electrodes.

Box for interaction with objects (BIO): A new device to synchronize the presentation of objects with electrophysiological recordings

Brain event-related potentials are a useful tool for investigating visual processing and action planning. This technique requires extremely accurate synchronization of stimulus delivery with recordings. The precision of the onset time of visual stimulus delivery is a major challenge when attempting to use real, three-dimensional objects as stimuli. Here, we present an innovative device, the “box for interaction with objects” (BIO), that is designed to synchronize the presentation of objects with electroencephalographic (EEG) recordings. To reach the required resolution of stimulus-onset timing, the BIO system features an interface with reflective glass and light-emitting diodes (LEDs). When the LEDs inside the BIO are turned on, the object inside becomes visible, and a synchronizing pulse is sent to the recording systems. The BIO was tested in a motivational study that focused on visual and motor event-related potentials. EEG signals were recorded during the presentation of an emotion-laden object that could be grasped and brought close to the participant’s chest. BIO successfully synchronized the appearance of a three-dimensional object with EEG recordings, which would allow for an analysis of visual and motor event-related potentials in the same experiment. The BIO device, through a high-quality psychophysiological approach, offers a new perspective for the study of the motivational factors that drive actions toward relevant stimuli.     

Wavelet analysis of neuroelectric waveforms: a conceptual tutorial

This paper presents a nontechnical, conceptually oriented introduction to wavelet analysis and its application to neuroelectric waveforms such as the EEG and event related potentials (ERP). Wavelet analysis refers to a growing class of signal processing techniques and transforms that use wavelets and wavelet packets to decompose and manipulate time-varying, nonstationary signals. Neuroelectric waveforms fall into this category of signals because they typically have frequency content that varies as a function of time and recording site. Wavelet techniques can optimize the analysis of such signals by providing excellent joint time-frequency resolution. The ability of wavelet analysis to accurately resolve neuroelectric waveforms into specific time and frequency components leads to several analysis applications. Some of these applications are time-varying filtering for denoising single trial ERPs, EEG spike and spindle detection, ERP component separation and measurement, hearing-threshold estimation via auditory brainstem evoked response measurements, isolation of specific EEG and ERP rhythms, scale-specific topographic analysis, and dense-sensor array data compression. The present tutorial describes the basic concepts of wavelet analysis that underlie these and other applications. In addition, the application of a recently developed method of custom designing Meyer wavelets to match the waveshapes of particular neuroelectric waveforms is illustrated. Matched wavelets are physiologically sensible pattern analyzers for EEG and ERP waveforms and their superior performance is illustrated with real data examples.     

类别归纳的时间过程和源定位 ——事件相关电位研究提供的电生理证据

采用事件相关电位（ERP）技术,以图形比较为实验作业探索类别归纳的神经机制。任务要求被试归纳两个图形刺激之间的共享特征,根据共享特征数量的不同构成三个任务：共享一个特征（S1F）、共享两个特征（S2F）和共享三个特征（S3F）。S1F任务和S2F任务被看作“归纳”任务,S3F任务被看作“非归纳”任务。对三个任务诱发的脑电分别进行叠加和平均,通过相减技术得到差异波（S1F－S3F,S2F－S3F）。在400到650ms时间窗口三个条件均诱发了LPC成分,地形图显示三个条件下LPC的头皮分布均出现在中－顶;两个“归纳”任务与“非归纳”任务在LPC上存在显著差异,而两个“归纳”任务在LPC上没有显著差异。对两个差异波在400到650ms时段的数据进行偶极子源分析,结果显示每个差异波均只有一个偶极子源,且二者定位相似（均在海马附近）。由于已知海马与新异联系的形成有关,因此LPC可能反映了类别归纳的关键加工——新异联结的形成     

高强迫倾向个体注意脱困的电生理证据

对威胁信息的注意偏向可能是导致和维持强迫症症状的重要因素之一。本研究采用情绪Stroop范式,同时结合ERP技术,对强迫倾向个体注意偏向的时间进程进行研究,以期探究强迫倾向个体注意偏向的神经机制特点。结果发现,高强迫倾向个体对强迫症相关消极词比一般消极词和中性词表现出了更大的P2和LPPs振幅。实验结果表明,高强迫倾向个体对强迫症相关消极词表现出注意偏向,其成分为注意脱困,既表现在早期自动加工阶段的P2成分上也表现在晚期策略加工阶段的LPPs成分上。     

Hippocampal electrical activity and behavior in rabbits living in a seminatural environment: reactions in the presence of a dog

The aim of the present experiment was to correlate behavioral and electrical events recorded in the presence of a threatening stimulus, namely a dog presented to rabbits living in a seminatural environment. Behavioral and concomitant EEG, recorded by telemetry, were studied in rabbits carrying chronic electrodes implanted in the hippocampus, before, during, and after the introduction of a dog into the environment around the enclosure. The threatening stimulus caused a variety of emotional responses in the rabbits, associated with high rhythmicity and high frequency levels in the hippocampal EEG. After removal of the dog the state of alarm of the animals persisted with long periods of alert immobility characterized by a further increase in hippocampal EEG frequency. Data show that emotional and environmental aspects can affect the hippocampal electrical activity recorded during specific behavioral elements.     

Effective use of the blank comparison procedure in simple discrimination by infant capuchin monkeys: A methodological note

In studies of simple and conditional discrimination, procedures are needed to measure those aspects of stimuli that control behavior. The blank comparison procedure is one such procedure. It was designed explicitly for assessing S+ and S- functions when discriminative stimuli are presented simultaneously. In this procedure, a neutral stimulus serves sometimes as S+ and sometimes as S-. Its discriminative function is defined in relation to other stimuli in the display. The present study aimed to prepare 2 infant female capuchin monkeys for the effective use of the blank comparison procedure in a simple discrimination task. First, simple discrimination training was applied up to a stable accuracy criterion of ≥90%. This training was followed by the replacement of S+ and then of S- stimuli with new stimuli. Ultimately, trials with the blank comparison were introduced. Following this sequence, both monkeys immediately displayed highly accurate blank-comparison performances without the need for stimulus control shaping or other preparatory discrimination training. Thus, this procedure sequence may be an efficient, effective method for establishing blank-comparison baselines for experimental analyses of S+/S- discriminative functions and perhaps for other applications in teaching simple and conditional discrimination performances to this species and others.     

Dealing with artifacts: The EOG contamination of the event-related brain potential

Eye movements and blinks represent a major source of artifacts in the electroencephalogram (EEG) and event-related brain potentials (ERPs). The origin of this artifact is the large difference in potential that exists between the cornea and the retina. Eye movements and blinks produce shifts of the electric fields that propagate across the whole head and that can be several times larger than the activity generated by the brain. Ocular activity can be monitored by electrodes located near the eyes (electrooculogram, or EOG). The electric fields associated with eye movements and blinks are somewhat different. The simplest procedure for dealing with ocular artifacts is to eliminate trials on which EOG activity is detected (rejection). However, this technique may result in data loss and biased data samples, especially when one is comparing clinical populations or tasks involving large amounts of eye movements. Another approach involves estimation and correction of the ocular artifact on the EEG and ERP traces. Several techniques have been proposed. Some of them are reviewed in the present paper. Issues related to the accuracy of the various techniques, as well as other advantages and limitations, are also discussed. Finally, general guidelines for how to deal with ocular artifacts are proposed.     

Cognitive neural prosthetics

Research on neural prosthetics has focused largely on using activity related to hand trajectories recorded from motor cortical areas. An interesting question revolves around what other signals might be read out from the brain and used for neural prosthetic applications. Recent studies indicate that goals and expected value are among the high-level cognitive signals that can be used and will potentially enhance the ability of paralyzed patients to communicate with the outside world. Other new findings show that local field potentials provide an excellent source of information about the cognitive state of the subject and are much easier to record and maintain than spike activity. Finally, new movable probe technologies will enable recording electrodes to seek out automatically the best signals for decoding cognitive variables.