P300-based Brain-Computer Interface Performance Enhancement

P300-based Brain-Computer Interface Performance EnhancementEnhancing Performance and Bitrates in a P300-based Brain-Computer Interface for disable Subjects by Phase-to-Amplitude Cross-Frequency CouplingStavros I.Dimitriadis1,2*, Avraam D. Marimpis31Institute of Psychological Medicine and Clinical Neurosciences, Cardiff University School of Medicine, Cardiff, UK2Cardiff University Brain Research Imaging Center, School of Psychology, Cardiff University, Cardiff, UK3Brain Innovation B.V., NetherlandsAbstractA brain- ready reckoner interface (BCI) is a converse outline that substitutes brain-natural process into specific commands for managing a information processing system or other home or electrical devices. In other words, a BCI is an alternative direction of interacting with the environment by using brain-activity instead of muscles and nerves. For that reason, BCI systems are of high clinical value for oriented populations suffering from neurological dis shapes. In this writ ing, we stage a in the buff treat approach of a well-known P300-BCI system for disable exits. Estimating cross-frequency mating (chlorofluorocarbon) and that is to say - grade-to-amplitude coupling (PAC) deep down sensor, we succeeded high sort accuracy and high bitrates for some(prenominal) change and fit subjects. The system is tested with four severely disabled and four able-bodied subjects. The bitrates obtained for both the disabled and able-bodied subjects r every(prenominal)ed the fastest account level of 10 bits/sec. The new preprocessing approach based on recordings from the single-sensor Pz while the classification accuracy is tested besides for others electrodes.Keywords Brain-computer interface P300 Disabled subjects cross-frequency coupling accuracy*Corresponding designerDr.Dimitriadis StavrosResearch FellowSchool of Medicine,Cardiff University, UKCUBRIC Neuroimaging Center, Cardiff,UKIntroductionFrom the very fasten-go bet of Farwell and Donchin 1 the bulk of P300-based Brain Computer Interface (BCI) systems foc utilize on developing new lotion scenarios 2,3, and on developing and testing new algorithms for the reliable detection of the P300 waveform from noisy datasets 4-8. For a review of P300, an interested toilette read the 9-10. 10 years ago, 2 pioneer studies guard been first published presenting a P300 BCI system on disabled subjects. Piccione et al. (2006) 11 design a 2D cursor BCI direct system w here subjects had to concentrate on four arrows flashing every 2.5 sec in stochastic order occupied the peripheral area of a computer screen. Five disabled and seven able-bodied subjects were participated on this experiment. For controlling this cursor, this four-choice P300 flashing arrow cursor was apply. encephalogram manifestations were recorded using four encephalogram sensors and electro-oculogram. utilize independent component analysis and neural networks, Piccione et al. 11 exhibit that P300 can be a valu able control signal for disabled subjects. However, the communication system was alike low compared to terra firma of the art systems 5,8.Sellers and Donchin (2006) 12 designed a four choice BCI experiment with four stimuli (YES, NO, PASS, END) that were presented every 1.4 s in random order, in two different modalities either visual or auditory or in a combined mode. Three subjects suffering with ALS and three able-bodied subjects performed the experiment. pneumoencephalogram recordings from three sensors were classified using a stepwise linear discriminant algorithm (LDA). They demonstrate that communication via a P300 system is possible for subjects suffering from ALS. Addition on the solelyy, they demonstrated that communication is possible in different modalities like the visual, auditory, and also in a combined auditory-visual mode. However, both the classification accuracy and the communication rate were low compared to the state of the art results. One of possible expla nations of low accuracy and communication rate could be the low number of EEG sensors, the long inter-stimulus intervals and the low number of trials.McCane et al., demonstrated a BCI system where both accuracy and communication rate did not differ significantly between ALS subprogramrs and HVs. Although ERP morphology was similar for the two groups, the target ERPs differed significantly in the status and amplitude of the late positivity (P300), the amplitude of the early negativity (N200), and the latency of the late negativity (LN) 13.Hoffmann et al.,, demonstrated a six-choice P300 double which was tested in a population of fivesome disabled and four able-bodied subjects. Six different ikons were flashed in random order with an ISI of cd ms 7. They tested how the electrode configuration can influence the accuracy in order to detect the best channel selection. For four out of five disabled subjects and for all the able-bodied subjects both the communication rates and the cl assification accuracies were higher compared to the aforementioned studies 11,12. The datasets in Hoffmann et al., study can be freely downloaded from the website of the EPFL BCI group (http//bci.epfl.ch/p300).In the present study, we used the dataset from Hoffmann et al., study to demonstrate an alternative algorithmic approach with main scope to improve the bitrates up to the limits. For that occasion, we adopted a cross-frequency coupling (CFC) estimator namely strain-to-amplitude coupling (PAC) to quantify how the phase of the set about frequency brain rhythms tone ups the amplitude of the higher oscillations. The whole approach was followed in a trial basis and within sensors located over parieto-occipital brain areas. PAC proved to be a valuable estimator in many applications like the design of a biomarker for amnesic mild cognitive impairment subjects during an auditory oddball paradigm 14, for dyslexia 15, for mild traumatic brain injury 16.The layout of the paper is as f ollows. In Section 2, we expound the subject population, the experiments that were performed, and the methods used for data pre-processing steps of the proposed pipeline and the classification procedure. Results are presented in Section 3. Discussion is devoted in Section 4.2. Materials and Methods2.1. Experimental frame-up Users were facing a laptop screen on which six images were displayed (see Fig. 1). The images showed a television, a telephone, a lamp, a door, a window and a radio. The images were selected according to an application scenario in which users can control electrical appliances via a BCI system. The application scenario served however only as an example and was not pursued in further detail. The images were flashed in random sequences, one image at a meter. Each flash of an image oddmented for 100 ms and during the following 300 ms none of the images was flashed, i.e. the inter-stimulus-interval was 400 ms.The EEG was recorded at 2048 Hz sampling rate from 32 el ectrodes placed at the standard positions of the 10-20 international system. A Biosemi Active Two amplifier was used for amplification and analog to digital conversion of the EEG signals.Figure 1 around here2.2. Subjects The proposed methodology was applied to P300 BCI-oriented recordings derived from five disabled and four healthy subjects. The demographics of the four disabled are presented in (Table 1). Disabled subject 5 as excluded from further analysis. Subjects 6-9 were Ph.D. students that were recruited from EPFL BCI groups laboratory (all males, age 302.3). None of subjects 6-9 had known neurological deficits. For more schooling regarding the subjects, an interested reader should refer to the original paper 7.Table 1. Subjects from which data was recorded in the study of the environment control systemS1S2S3S4Diagnosis cerebral palsyMultiple sclerosisLate-stageamyotrophiclateral sclerosisTraumatic brainand spinal-cordinjury, C4 levelAge56514733Age at illness onset0 (perinat al)373927sSexMMMFSpeech productionMild dysarthriaMild dysarthria sodding(a) dysarthriaMild dysarthriaLimb muscle controlWeakWeakVery WeakWeakRespiration control everydayNormalWeakNormalVoluntary eye movementNormalMild nystagmusNormalNormal2.3. Experimental schedule Each subject completed four recording sessions. The first two sessions were performed on one day and the last two sessions on another day. All subjects were recruited within maximum two weeks between the first and the last session. Each of the sessions consisted of six happens, one run for distributively of the six images. For further details about the protocol followed on this experiment see the original paper related to this dataset 7.The following protocol was used in each of the runs.(i) Subjects were asked to count silently how often a prescribed image was flashed (for example Now cheer count how often the image with the television is flashed).(ii) The six images were displayed on the screen and a warning tone was issued.(iii) Four seconds after the warning tone, a random sequence of flashes was started and the EEG was recorded. The sequence of flashes was block-randomized, this marrow that after six flashes each image was flashed once, after twelve flashes each image was flashed twice, etc. The number of blocks was elect randomly between 20 and 25. On honest 22.5 blocks of six flashes were displayed in one run, i.e. one run consisted on average of 22.5 target (P300) trials and 22.5-5 = 112.5 non-target (non-P300) trials.(iv) In the second, third, and fourth part session the target image was inferred from the EEG with a simple classifier. At the end of each run the image inferred by the classification algorithm was flashed five times to give feedback to the user.(v) After each run subjects were asked what their counting result was. This was done in order to monitor performance of the subjects.The duration of one run was approximately one minute and the duration of one session including se tup of electrodes and short breaks between runs was approximately 30 min. One session comprised on average 810 trials, and the whole data for one subject consisted on average of 3240 trials.2.4 Offline AnalysisThe impact of different single-sensor recordings on classification accuracy was tested in an offline procedure. For each subject four-fold cross-validation was used to estimate average classification accuracy. More specifically, the data from three recording sessions were used to train a classifier and the data from the left-out session was used for validation. This procedure was repeated four times so each session served once for validation.2.4.1. PreprocessingBefore learning a classification function and before validation, some(prenominal) preprocessing operations were applied to the data.The preprocessing operations were applied in the order stated below.Referencing. The average signal from the two mastoid electrodes was used for referencing.Filtering. A third order forwar d-backward Butterworth bandpass filter was used to filter the data.. The MATLAB function butter was used to compute the filter coefficients and the function filtfilt was used for filtering. The predefined frequencies were 0.5-4 Hz, 4-8 Hz,1 8-10 Hz, 2 10-13 Hz,1 13-20 Hz,2 20-30 Hz and 1 30 45 Hz.(iii) Downsampling. The EEG was down-sampled from 2048 Hz to 512 Hz by selecting each quaternary sample from the bandpass-filtered data.(iv) Single trial extraction. Single trials of duration 1000 ms were extracted from the data. Single trials started at stimulus onset, i.e. at the beginning of the intensification of an image, and ended 1000 ms after stimulus onset. Due to the ISI of 400 ms, the last 600 ms of each trial were overlapping with the first 600 ms of the following trial.(v) Electrode selection. We applied our analysis to recordings from single-sensor activity and mainly, PZ,OZ,P3,P4,P7 and P8.(vi) Feature vector construction. As appropriate make for each trial, we used the p hase-to-amplitude coupling (PAC) which already has been shown its potentiality in building reliable biomarkers (Dimitriadis et al., 2015,2016). PAC was estimated for each frequency pair (see ii)). The description of PAC is given in the next section. As a complementary feature that can separate the counted stimuli from the non counted stimuli, alpha sexual intercourse signal powers have been estimated. Alpha power level can give us a valuable and objective criterion when a subject attends or not attends to the stimulus. Our idea is to create an initial binary classifier that will cut-off the attended from the non-attended stimuli for each subject prior entering the main multi-class classifier.CFC metric numerationCFC estimates the strength of pairwise interactions and identifies the prominent interacting pair of frequencies, both between and within sensors 17-19. Among available CFC descriptors, phase-amplitude coupling (PAC), which relies on phase coherence, is the one most unrem arkably encountered in research 20. The PAC algorithm as adapted to continuous MEG multichannel recordings is described below.he within-sensor CFC version is described. Let x(isensor, t), be the EEG activity recorder at the isensor-th site, and t=1, 2,. T the successive time points. Given a frequency-limited signals x(isensor,t) , cross-frequency coupling is estimated by allowing the phase of the lower frequency (LF) oscillations to modulate the amplitude of the higher frequency (HF) oscillations. The complex analytic representations of each signal zLF(t) and zHF(t) are derived via the Hilbert transform (HT.).Next, the envelope of the higher-frequency oscillations AHF(t) is bandpass-filtered within the range of LF oscillations and the resulting signal is submitted to an additional Hilbert transform to derive its phase dynamics component (t)which expresses the modulation of the amplitude of HF-oscillations by the phase of the LF-oscillations. Phase consistency between the two time-se ries was measured by means of both the original definition 21 and the speculative portion of PLV, as synchronization indexes to quantify the strength of PAC.The original PLV is defined as followsand the imaginary part of PLV as followsThe imaginary portion of PLV is considered to be less susceptible to volume conduction effects in assessing CFC interactions. While the imaginary part of PLV is not touch by volume conduction effects, it could be sensitive to changes in the angle between two signals, which not necessarily imply a PLV change. In general, the imaginary portion of PLV is only sensitive to non-zero-phase lags and is thus resistant to instantaneous self-interactions associated with volume conductance 22. For further details and applications, an interested reader can read our previous work 14,15.In the present study, as was already mentioned we used 8 frequency bands which means that PAC is estimated for 7*6/2=21 cross-frequency pairs e.g. A , 1Awhere and A denote the phase and amplitude of each frequency band. Figure 2 demonstrates the pre-processing steps of the PAC estimator for a trial of subject 6 at target image 6.Figure 2 around hereSignal PowerWe estimated the proportional power of each band-pass frequency signal segment with the following comparabilitysThe first equation quantifies the signal power (SP) of each frequency as the sum of the filtered signal squared per sample (3) while equation (4) divides the SP by the sum of the SP from all the frequencies which gives the coition signal power (RSP). The whole approach was repeated for every trial, sessions and subject.2.4.4. Machine learning and classificationTraining data sets contained 405 target trials and 2025 non-target trials and validation data sets consisted of cxxxv target and 675 non-target trials (these are average values cf. Section 2.3).Adopting sequential feature selection algorithm, we detected the characteristic cross-frequency pair via PAC value that gives the highest contrast of each target images compared to the rest based on the training data set. Additionally, we used the same feature extraction algorithm to detect the relative signal power that separate the counted flashing images from the non-counted images.We trained a multi-class SVM classifier based on the selected PAC estimate from specific cross-frequency pairs and then we tested the classifier to the validation data to get the response tailored to each target image 23. The training test consisted of the first session while the rest three sessions were used for validating the whole analytic scheme. A k-nearest neighbour (k-NN) classifier was applied to differentiate the attended from the non-attended flashing images prior to multi-class SVM classifier.2.4.5 Performance EvaluationClassification accuracy and ITR were calculated for the offline experiments separately. The method for reckon ITR (in bits per second) was as follows (5)Where N is the number of classes (i.e., 6 in this study ), P is the accuracy of target identification, and T (seconds per selection) is the average time for a selection.Results- Phase-to-Amplitude Coupling as a Valuable Feature for the BCI-P300 SystemWe estimated both PAC and relative signal power (RSP) for the first 32 samples (60 ms) increasing the window up to 500 ms (256 samples) with a step of 12 samples (5 ms) . The sequential feature selection algorithm detected only one PAC feature from the 21 possible cross-frequency pairs as the unique candidate feature to separate the six classes of images-stimuli. A was the selected feature for both disabled and able-bodied subjects.The group-averaged classification performance was for each sensor location using the first 100 ms for both able-bodied and disabled subjects. The errors were detected on the trials where the subject missed the flashing image. The classification performance with the use of a kNN-classifier prior to the multi-class SVM was 100 % for every subject and for all the pre-selected sensors namely PZ,OZ,P3,P4,P7,P8 EEG sensors. Figure 3 and Figure 4 illustrates the trial-related (grand-averaged) PAC-connectivity patterns (comodulograms) for subject 6 (able-bodied) and subject 1 (disabled) correspondingly from target and non-target trials for each flashing image. Comodulograms differed by contrasting target vs non-target within each subject and target image but also between the two images. A was the unique feature for both disabled and able-bodied subjects that can clearly predict the target image for both groups.Figures 3 and 4 around hereAttention and Alpha PowerPrior to multi-class SVM, we applied a kNN-classifier based on 1 signal power which was selected as the feature that can discriminate counted from non-counted flashing images. The kNN-classifier performed 100 % clear filtration of attended from non-attended trials for each subject and further amend the performance of multi-class SVM to 100 %. We achieved this performance using 1 signal relative power estimated from the first 100 ms for both able-bodied and disabled subjects.The classification performance with the kNN-classifier was 100 % for every subject and for all the pre-selected sensors namely PZ,OZ,P3,P4,P7,P8 EEG sensors.Table 2 summarizes the group-averaged relative signal power (RSP) of 1 frequency band for attended versus non-attended images.Table 2. Group-averaged 1 signal relative power for attended and non-attended images.AttendedNon-AttendedAble BodiedDisabledPerformance EvaluationIn the present study, we succeeded bitrates of 10 bits/sec for both disabled and able-bodied subjects correspondingly for all the sensor locations used on the whole analysis. The time for estimation of PAC and testing the trial was 0.00001 sec on a Windows 7 -Intel 7 8-core machine.DiscussionA novel approach of how to analyse single-trials in a BCI system was introduced based on the estimation of cross-frequency coupling (CFC) and namely phase-to-amplitude coupling (PAC). PAC was estimated within EEG sensors from single-trials recorded during a visual evoked experimental paradigm. The proposed analytic scheme based on the extraction of unique features from the CFC patterns on a single-trial basis and namely the A coupling, as a unique feature for both able-bodied and disabled subjects. Our experimentations showed a high classification rate (99.7%) based on the proposed PAC feature. Additionally, the superiority of our approach compared to alternative popular methodologies like the use of the original recordings was evident from the succeeded bitrates (10 bits/sec) and also of the response time of the classification system (0.00001 sec). Complementary, using a binary classifier trained with 1 relative signal power prior to the multi-class SVM, we differentiated the attended from the non-attended stimuli which further improved the classification performance up to 100% in both groups.Compared to many other P300-based BCI systems designed for disabled users, we succeeded the highest classification accuracy and bitrates higher than the original presented dataset 7. In previous studies like the one of Sellers and Donchin (2006) 12, the best classification accuracy for the able-bodied and ALS subjects was on average 85% and 72% correspondingly 12. Hoffmann et al., succeeded living classification accuracy for both disabled and able bodied subjects for the first demonstration of the current dataset. However, he used longer time series of over 15-20 secs by concatenating trials in order to train better the classifier. Additionally, he used one classifier per image per each of the twenty block and the final outcome derived as the majority voting of the twenty classifiers. Here, using phase-to-amplitude coupling as the appropriate descriptor of the evoked response in the parietal brain areas and a multi-class SVM classifier, we performed almost absolute accuracy (99.97) on a trial basis. Using an additional binary k-NN classifier and 1 relative signal power prior to multi-class SVM, we separated the attended (counted) from the non-attended (not counted) trials guide to zeroing the misclassified trials from the multi-class SVM classifier for every subject. This procedure further improved the classification performance from 99.97 to 100% for each subject. We succeeded bitrates faster (10 bits/sec) than any other BCI system including the fastest spelling system presented recently (5.32 bits/sec 24). In a previous study of Piccione et al. (2006) 11 average bitrates of about 8 bits/min were reported for both disabled and able-bodied subjects. Hoffman et al., 2008 7 reported average bitrates obtained with electrode configuration (II) (8 electrodes) 12.5 bits/min for the disabled subjects and 10 bits/min for the able-bodied subjects.According to Klimeschs theory, on the early stages of perception, directs the data flow towards to neural substrates that represent information relevant for the encoding system (e.g. vis ual stimulus to visual system, voice/sound to auditory system). The physiological main function of is linked to inhibition. Klimeschs theory hypothesizes that enables to have access to stored information by inhibiting task-irrelevant neuronal substrates and by timing/synchronizing the cortical activity in task-relevant neuronal systems. A lot of research findings showed that both evoked and phase locking are evidence of a successful encoding of global stimulus features in an early post-stimulus interval of about 0-150ms 25.Besides the cross low-frequency/high-frequency coupling (e.g.,- 26,27), there are many evidences 28-31 that CFC exists also between the low-frequency bands (e.g., delta-theta, delta-alpha, and theta-alpha). Lakatos et al. (2005) 29 introduced a possible action about the hierarchical organization of EEG oscillations suggesting that the amplitude of the brain oscillations at a characteristic frequency band can be modulated by the oscillatory phase at lower freq uency. In particular, they found that (1-4 Hz) phase modulates (4-10 Hz) amplitude, and modulates (30-50 Hz) amplitude in primary auditory cortex of awake macaque monkeys 29. This multiplex coupling or nesting of brain rhythms might ring a general brain organizational principle, as evidence of coupling (mainly -) has also been observed in animals (e.g. rats,cats) and humans 32.For instance, in auditory cortex, -band modulates the amplitude of -band ICMs, whose phase in turn modulates the amplitude of -band ICMs 33. This indirect enhancement effect uses the ongoing activity of local neural activity in the primary auditory cortex. Their hypothesis supports the notion that neural oscillations reflect rhythmic shifting of excitability states of neural substrates between high and low levels. This hypothesis is supported by the fact that oscillations can be predicted by visual input such as the auditory input arrives during a high excitability phase and is amplified. In the present s tudy, we demonstrated that (0.5-4 Hz) phase modulates (4-8 Hz) amplitude over visual brain areas due to flashing images, their content and mainly was observed on parietal EEG recording sites.We should also mention that the reason why A coupling discriminates the six flashing images can be directly linked to the content of the images. Visual attention samples image stimuli rhythmically demonstrating a peak of phase at 2 Hz 34 while flashing images induced rhythmic fluctuation at higher frequencies (6-10 Hz) 35 here within frequency range 4-8 Hz.Finally, the work of Karakas et al., (36) showed that the ERP represents interplay between the oscillations that are mainly in the and frequencies and directly linked to P300 37.ConclussionIn this work, an efficient algorithmic approach was presented to a P300-based BCI system for disabled subjects. We have shown that absolute classification accuracies and the highest reported bitrates can be obtained for severely disabled subjects unde r the notion of cross-frequency coupling and namely phase-to-amplitude coupling. Specifically, (0.5-4 Hz) phase modulates (4-8 Hz) amplitude proved to be the candidate feature from PAC estimates that supported the highest classification accuracy, the fast bitrates and the fast response time of the multi-class system. Due to the use of the P300, only a small amount of training (trials from 1st session as a training set and 100ms per trial) was required to achieve good classification accuracy. future improvements to the work presented could be the design of useful BCI applications adapted to the needs of disabled users. Also it might be useful to perform exploratory analysis on larger populations and on real-time to further validate the results found in the present work.AcknowledgementsSID was supported by MRC grant MR/K004360/1 (Behavioural and Neurophysiological Effects of dementia praecox Risk Genes A Multi-locus, Pathway Based Approach)ReferencesFarwell LA, Donchin E. Talking o ff the top of your head toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 198870510-23.Polikoff J, Bunnell H, Borkowski W. Toward a P300-based computer interface. In Proceedings of the RESNA95 Annual Conference 1995.Bayliss JD. Use of the evoked P3 component for control in a virtual apartment. IEEE Trans Neural Syst Rehab Eng 200311(2)113-6.Xu N, Gao X, Hong B, Miao X, Gao S, Yang F. BCI competition 2003 Data fix IIb Enhancing P300 wave detection using ICA-based subspace projections for BCI applications. IEEE Trans B