People suffering from neurodegenerative disorders, such as Amyotrophic Lateral Sclerosis (ALS), can eventually present great disabilities. In some cases, these patients lose all possibility to communicate with the external world via common muscular channels, being the only alternative the use of a Brain-Computer Interface (BCI) system, which transforms brain activity into external commands. A P300-speller is a typical Brain-Computer Interface system for communication purpose. In order to facilitate the communication, it is very important to adapt the speller to each patient. The most popular platforms to develop P300 speller are BCI2000, OpenVibe and UMA-BCI Speller. The goal of this study was to evaluate the usability of the three proposed platforms in terms of effectiveness, efficiency and satisfaction. To this end, three participants had to configure a specific speller layout using the 3 platforms. The obtained results indicated that the UMA-BCI Speller platform presented the highest level of usability, following by the BCI2000 and finally, the OpenVibe platform. In this sense, the UMA-BCI Speller seems to be an easy application to use, providing many options and allowing to configure any speller layout in an easy way.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Ron-Angevin, Ricardo

English

Repositorio Institucional Universidad de Málaga

Usability Study of Different Platforms to Develop Communication Systems Based on P300-Brain-Computer Interface (BCI)   Ron-Angevin R. Departamento de Tecnología Electrónica Universidad de Málaga Malaga, Spain e-mail: rron@uma.es   Abstract—People suffering from neurodegenerative disorders, such as Amyotrophic Lateral Sclerosis (ALS), can eventually present great disabilities. In some cases, these patients lose all possibility to communicate with the external world via common muscular channels, being the only alternative the use of a Brain-Computer Interface (BCI) system, which transforms brain activity into external commands. A P300-speller is a typical Brain-Computer Interface system for communication purpose. In order to facilitate the communication, it is very important to adapt the speller to each patient. The most popular platforms to develop P300 speller are BCI2000, OpenVibe and UMA-BCI Speller. The goal of this study was to evaluate the usability of the three proposed platforms in terms of effectiveness, efficiency and satisfaction. To this end, three participants had to configure a specific speller layout using the 3 platforms. The obtained results indicated that the UMA-BCI Speller platform presented the highest level of usability, following by the BCI2000 and finally, the OpenVibe platform. In this sense, the UMA-BCI Speller seems to be an easy application to use, providing many options and allowing to configure any speller layout in an easy way.  Keywords- Brain-Computer Interfaces (BCI); P300 speller; Usability; Amyotrophic Lateral Sclerosis (ALS)   I. INTRODUCTION People suffering from neurodegenerative disorders, such as Amyotrophic Lateral Sclerosis (ALS), can eventually present great disabilities, particularly those involving the motor system. In some cases, such deficiencies can be really severe, to the point of causing total loss of control of the muscles that are responsible of voluntary body movements, including eye movement and breathing itself. People suffering from such disabilities lose all possibility to communicate with the external world via common muscular channels. Their only alternative is to use a Brain-Computer Interface (BCI) system [1], which transforms brain activity into commands that are interpreted by a machine. Such a system offers a non-muscular channel for these users to interact with their environment, thus providing them with greater autonomy in their daily lives. The most widely used BCI systems are those based on electroencephalographic (EEG) signal recording, due to its non-invasiveness, but also to its good temporal resolution and ease of use. Three types of EEG-based BCI systems have been used for communication purposes, namely those based on: (a) slow cortical potentials (SCPs), (b) P300 event-related potentials (ERP), and (c) sensorimotor rhythms (SMR) [2]. BCIs based on SCP and SMR demand that users are extensively trained before they show sufficient control of their brain activity. In contrast, BCIs based on P300 rely on a common, expected human response to infrequent target stimuli—usually visual—and thus require minimal training. The P300 signal, recorded over the central and parietal regions, is a positive deflection of brain wave at a latency of about 300 ms after stimulus presentation. The main applications of P300-based BCI systems are aimed at communication purposes. They are based on the P300 speller first developed by Farwell and Donchin [3], which is still referenced and intensely studied. In this BCI, a 6 x 6 matrix of letters, arranged in rows and columns, is shown to the subject. The user focuses his/her attention on the matrix element he/she wishes to select as each row and column is flashed (i.e., intensified) randomly, one after the other. After a number of flashes, the symbol that the user was supposedly attending at is presented on screen. In order to study variations and alternative paradigms, it is very important to be able to configure the different elements of the speller, such as the size, color, characters, images, etc. Besides, a configurable speller would be adapt to each patient to facilitate the communication. The most popular platforms to develop P300 speller are BCI2000 [4] and OpenViBE [5]. Both are widely used, with up-to-date software releases, documentation and support. These two platforms are intended to build end user BCI applications, however, they still require technical skills in order to implement a P300 speller. As both are general-purpose platforms, with a high degree of configurability, it may be complex to parameterize them in order to obtain the desired speller. Recently, the BCI research group of the University of Malaga (UMA-BCI) decided to implement a BCI system platform based on a P300 speller, which is easy to use and flexible enough to configure any spellers. This platform is called UMA-BCI Speller [6]. The official ISO 9241-11 definition of usability is: “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” This definition involves three measures: i) effectiveness (i.e., accuracy and completeness of the system with which users achieve set goals), ii) efficiency (i.e., resources expended to complete goals), and iii) satisfaction (i.e., users’ attitude to complete a given task) [7] [8]. The goal of this study was to evaluate the usability of the three proposed platforms in terms of effectiveness, efficiency and satisfaction. The study was focused on the feasibility of the three platforms to change the speller layout, and not to configure the signal acquisition and processing. In fact, signal acquisition and processing changes are less frequent that changes in layout interface, which are necessary to adapt the communication systems to the patients. The obtained results will allow researchers to select the most appropriate platform to develop communication systems based on P300 spellers for people with serious motor function problem, such as, patients suffering from ALS. The rest of the paper is structured as follows. Section 2 presents the methodologie and the material used. Section 3 presents the obtained results abd the discussion and, finally, section 4 presents the conclusion of the study.  II. MATERIAL AND METHODS In this section, a description of the experiment carried out to evaluate the usability of the three platforms will be described.  A.  Experimental design The objective was to evaluate the feasibility to change the configuration of the P300 speller interface, then, it was necessary to check the different options available on each platform in order to set the configuration of the final speller. Once established the final speller layout, the participants had to transform the speller of reference (the same than the one first developed by Farwell and Donchin, see Figure 1) into the final proposed speller. Some days before the test, a manual of each platform was provided to the subjects. These manuals were specially made for this experiment and included only instruction regarding how to make the different changes concerning the speller layout.   Figure 1.  Common P300 speller matrix proposed by Farwell and Donchin. It consists of a 6 x 6 matrix of grey letters and numbers and black background.  Three subjects participated in this study. None of them had previous experience with any of the systems and were  novices to BCI. The three platforms were tested by each participant in three different sessions carried out on the same day. The time interval between sessions was, at least, one hour. The order in which the platforms were tested was different for each participant.  B. Advantages and restriction of each platform  As it was mentioned, in order to propose a final speller layout to carry out with each platform, it was necessary to test the different options provided by each platform. The study of each platform allowed, not only to establish the advantages and the restriction of each platform, but also to make the manuals for each platform.  Table 1 summarizes the viability to perform different changes in the speller layout for each platform. Each option was classified between difficult (D), moderate (M) and easy (E).   C. Task and procedure  Taking into account the different options provided by each platform to modify the speller layout, the final layout proposed in the experiments is shown in Figure 2.    TABLE I.  VIABILITY TO PERFORM DIFFERENT CHANGES IN THE SPELLER LAYOUT FOR EACH PLATFORM   UMA-BCI Speller OpenVibe BCI2000 Modifying the speller matrix size E M E Modifying the color of the background E E E Modifying the color of each cell E D D Choosing a specific color E E D Changing size and type of characters E M M Replacing characters by images E D M    Figure 2.  P300 speller proposed. For each platform, the users were asked to change the element´s layout to modify the P300 speller of reference into this one.  After reading the manual, the user had to transform the speller of reference into the proposed speller. This transformation meant to carry out some changes: i) the matrix size (to 4x4), ii) the color of the background (to purple), iii) the inclusion of some images, iv) the configuration of some characters (“A”, “B”, “C”, “D”, “1”, “2”, “3”, “4”) and v) the color of some cells. During the experiments, the users could consult the manual provided and did not receive any instruction regarding which steps to follow regarding the different changes. For each platform, the time available to perform the task was 60 minutes. Once finishing the task, the users were asked to complete different questionnaires to evaluate the usability.  D. Objectives and subjective measures As mentioned in the introduction section, the main objective of this study was to evaluate the usability of the different platform.   TABLE II.  PERCENTAGE OF CORRECT CHANGES (IN %) CARRIED OUT FOR EACH PLATFORM AND USER   User 1 User 2 User 3 UMA-BCI Speller 100 100 100 OpenVibe 80 100 100 BCI2000 100 100 100  The employed usability approach includes three dimensions: effectiveness, efficiency and satisfaction. Effectiveness is related to the accuracy with which a user can complete tasks. In order to study the effectiveness, the percentage of correct steps or changes were evaluated. Efficiency is related to the resources expended to complete a task, i.e., user´s effort and time required. In order to study the efficiency, the following metrics were provided: i) the subjective workload assessed using NASA-TLX [9]; ii) the time required to complete the task. Finally, satisfaction is related to the users’ attitude, i.e., the perceived comfort and acceptability while using the system. This dimension was evaluated through a subjective questionnaire regarding the use of each platform. The questionnaire consisted on 5 statements ranging from 1 to 5: statement S1: the platform is unnecessarily complex, statement S2: I would need external help to use the platform, statement S3: the platform seemed to me tedious to use, statement S4: It was necessary to have many knowledge before to use the platform, statement S5: the platform was easy to use. Statements S1 to S4 correspond to negative questions and the range was identified by: 1= completely disagree and 5 = completely agree. However, statement S5 correspond to a positive question and the range was identified in an opposite way, i.e.: 1= completely agree and 5 = completely disagree. In this sense, for all statements, a low value is always favourable.  III. RESULTS AND DISCUSSION Table 2 summarizes the obtained results for the effectiveness dimension: percentage of correct changes.     TABLE III.  TOTAL WORKLOAD AND TIME (IN MIN) REQUIRED TO COMPLETE THE TASK    User 1 User 2 User 3 Mean UMA-BCI Speller Workload 14,3 28,6 23 21,9  time 5 4 6 5 OpenVibe Workload 64,6 72 96 77,5  time 60 38 45 47,6 BCI2000 Workload 51,6 52,6 68 57,4  time 20 18 22 20  TABLE IV.  SCORES OF ANSWERS REGARDING THE SATISFACTION DIMENSION FOR EACH PLATFORM AND USER   User (U) Statements (S) S1 S2 S3 S4 S5  1 1 1 1 1 1 UMA-BCI Speller 2 1 1 1 2 1  3 1 1 1 2 1  1 2 4 5 5 5 OpenVibe 2 4 4 2 4 5  3 4 4 5 4 5  1 2 2 2 3 3 BCI2000 2 3 3 3 4 3  3 3 4 4 4 2    Regarding the efficiency dimension, Table 3 shows the contribution of the total workload (NASA-TLX global score ranged from 0 to 100) provoked by the use of the platform and the time required to complete the task. Finally, regarding the satisfaction dimension, the answers given by the participants at the end of each session in the usability questionnaire related to the use of the platforms are shown in Table 4. All users achieved to carry out all the changes in the speller layout with the three platforms, except user 1 with the OpenVibe platform. This user was able to make only 80% of the correct changes being, the lack of time, the main reason to make all the changes. Effectively, as it can be observed in Table 3, user 1 used up, for the OpenVibe platform, the maximum time available for the test (60 min). The time and total workload required to complete the task are metrics to measure the efficiency dimension. Users required, on average, only 5 min to complete the task with the UMA-BCI Speller platform, being 20 min and 47,6 min for BCI2000 and OpenVibe platforms, respectively. These times are in concordance with the total workload required for users, being low the subjective workload with the UMA-BCI Speller platform (average: 21,9), high with the OpenVibe platform (average: 77,5) and moderate with the BCI2000 platform (average: 57,4). According to these results, the UMA-BCI Speller platform could be denoted as the most efficient. Satisfaction has been studied according to six dimensions (or statements): complex, help required, tedious, knowledge necessary and easy. Each statement was ranging from 1 to 5, being the score “1” the most favourable. According to Table 3, the UMA-BCI Speller was, once again, the platform with the best scores. Except for statement S4 for user 2 and 3 with a score of “2”, all the statements were scored with “1”, being the average score 1,13. The average scores between users and statements was 4,13 and 3 for OpenVibe and BCI2000 platform respectively. These results showed that the UMA-BCI Speller platform was the most satisfactory and the OpenVibe platform, the less. This conclusion could be obtained according to statement S5: the platform was easy to use. All users considered the OpenVibe platform very difficult to use (score 5), however they considered the UMA-BCI Speller platform very easy to use (score 1), being the difficulty of the BCI2000 platform, moderate.  IV. CONCLUSION A P300 speller is a communication system controlled by brain activity, being, for some patients, the only option to communicate with the external world. To this end, it is very important to adapt the speller layout to each patient. In this sense, the available platform to configure a P300 speller should be easy to use. Despite there are several platforms to developed P300 spellers, frequently, users do not have information about which platform is more appropriate and easy to use. The present work has studied the usability of the three most popular platforms to develop P300 speller: BCI2000, OpenViBE and UMA-BCI Speller. Despite the low number of users, the obtained results allow to order the three platforms according to their degree of usability, considering the effectiveness, efficacy and satisfaction dimensions. The UMA-BCI Speller, recently developed by our team at the University of Málaga, offered the best scores in all dimensions, being the platform with the highest level of usability. The OpenVibe platform was difficult to use for naïve users, and then with a low level of usability. These results will allow, not only to the community researchers, but also to the final users, to select the most appropriate platform for developing BCI spellers as adapted as possible to the needs of each patient. In the future, we plan to increase the number of subjects in order to validate the obtained results. Besides, it should be interesting to compare the usability of each platform, not only for novices, but also for experienced users. ACKNOWLEDGMENT This work was partially supported by the project SICCAU: RTI2018-100912-B-100 (MCIU/AEI/FEDER, UE) and by the University of Malaga (“Universidad de Málaga”). Moreover, the author would like to thank Selene Moreno Calderon for her cooperation.  REFERENCES  [1] J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M.Vaughan, “Brain-computer interfaces for communication and control,” Clinical Neurophysiology, vol. 113, no. 6, pp. 767–791, 2002.  [2] J. Mak and J. R. Wolpaw, “Clinical applications of brain-computer interfaces: Current state and future prospects,” IEEE Reviews in Biomedical Engineering, vol. 2, pp. 187–199, 2009. [3] L. A. Farwell and E. Donchin, “Talking off the top of your head: toward a mental prothesis utilizing event-relatedpotencials,” Electroencephalography Clinical. Neurophysiology, vol. 70, pp. 510–523, 1988. [4] G. Schalk, D. J. McFarland, T. Hinterberger, N. Birbaumer, and J. R. Wolpaw, “BCI2000: A general-purpose brain-computer interface (BCI) system,” IEEE Trans. Biomed. Eng., vol. 51, no. 6, pp. 1034–1043, 2004. [5] C. Arrouët et al., “Open-ViBE: A three dimensional platform for real-time neuroscience,” Journal of Neurotherapy., vol. 9 no. 1, pp. 3–25, 2005. [6] F. Velasco-Álvarez et al., “UMA-BCI speller: an easily configurable P300 speller tool for end users,” Computer. Methods and Programs in Biomedicine, vol. 172, pp. 127–38, 2019. [7] J. Nielsen, “What Is Usability?,” in: User Experience Re-Mastered, pp. 3–22, 2010. [8] J. Nielsen and R. Molich, “Heuristic Evaluation,” in: Usability Inspection Methods, 1994. [9] S. G. Hart and L. E. Staveland, “Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research,” Advances in Psychology, vol. 52, pp. 139–183, 1988.     

Usability study of different platforms to Develop Communication Systems based on P300-Brain-Computer Interface (BCI).

https://riuma.uma.es/xmlui/bitstream/10630/20116/1/Camera_ready_BRAININFO2020_RON_ANGEVIN_v2.pdf

Usability study of different platforms to Develop Communication Systems based on P300-Brain-Computer Interface (BCI).

Abstract

Similar works

Full text

Available Versions

Repositorio Institucional Universidad de Málaga