Using Variable Natural Environment Brain-Computer Interface Stimuli for Real-time Humanoid Robot Navigation

This paper addresses the challenge of humanoid robot teleoperation in a natural indoor environment via a Brain-Computer Interface (BCI). We leverage deep Convolutional Neural Network (CNN) based image and signal understanding to facilitate both real-time bject detection and dry-Electroencephalography (EEG) based human cortical brain bio-signals decoding. We employ recent advances in dry-EEG technology to stream and collect the cortical waveforms from subjects while they fixate on variable Steady State Visual Evoked Potential (SSVEP) stimuli generated directly from the environment the robot is navigating. To these ends, we propose the use of novel variable BCI stimuli by utilising the real-time video streamed via the on-board robot camera as visual input for SSVEP, where the CNN detected natural scene objects are altered and flickered with differing frequencies (10Hz, 12Hz and 15Hz). These stimuli are not akin to traditional stimuli - as both the dimensions of the flicker regions and their on-screen position changes depending on the scene objects detected. On-screen object selection via such a dry-EEG enabled SSVEP methodology, facilitates the on-line decoding of human cortical brain signals, via a specialised secondary CNN, directly into teleoperation robot commands (approach object, move in a specific direction: right, left or back). This SSVEP decoding model is trained via a priori offline experimental data in which very similar visual input is present for all subjects. The resulting classification demonstrates high performance with mean accuracy of 85% for the real-time robot navigation experiment across multiple test subjects.Comment: Accepted as a full paper at the 2019 International Conference on Robotics and Automation (ICRA

Machine Vision-based Obstacle Avoidance for Mobile Robot

Obstacle avoidance for mobile robots, especially humanoid robot, is an essential ability for the robot to perform in its environment. This ability based on the colour recognition capability of the barrier or obstacle and the field, as well as the ability to perform movements avoiding the barrier, detected when the robot detects an obstacle in its path. This research develops a detection system of barrier objects and a field with a colour range in HSV format and extracts the edges of barrier objects with the FindContoure method at a threshold filter value. The filter results are then processed using the Bounding Rect method so that the results are obtained from the object detection coordinate extraction. The test results detect the colour of the barrier object with OpenCV is 100%, the movement test uses the processing of the object's colour image and robot direction based on the contour area value> 12500 Pixels, the percentage of the robot making edging motion through the red barrier object is 80% and the contour area testing <12500 pixel is 70% of the movement of the robot forward approaching the barrier object

Machine Vision-based Obstacle Avoidance for Mobile Robot

Obstacle avoidance for mobile robots, especially humanoid robot, is an essential ability for the robot to perform in its environment. This ability based on the colour recognition capability of the barrier or obstacle and the field, as well as the ability to perform movements avoiding the barrier, detected when the robot detects an obstacle in its path. This research develops a detection system of barrier objects and a field with a colour range in HSV format and extracts the edges of barrier objects with the FindContoure method at a threshold filter value. The filter results are then processed using the Bounding Rect method so that the results are obtained from the object detection coordinate extraction. The test results detect the colour of the barrier object with OpenCV is 100%, the movement test uses the processing of the object's colour image and robot direction based on the contour area value> 12500 Pixels, the percentage of the robot making edging motion through the red barrier object is 80% and the contour area testing <12500 pixel is 70% of the movement of the robot forward approaching the barrier object

On Tackling Fundamental Constraints in Brain-Computer Interface Decoding via Deep Neural Networks

A Brain-Computer Interface (BCI) is a system that provides a communication and control medium between human cortical signals and external devices, with the primary aim to assist or to be used by patients who suffer from a neuromuscular disease. Despite significant recent progress in the area of BCI, there are numerous shortcomings associated with decoding Electroencephalography-based BCI signals in real-world environments. These include, but are not limited to, the cumbersome nature of the equipment, complications in collecting large quantities of real-world data, the rigid experimentation protocol and the challenges of accurate signal decoding, especially in making a system work in real-time. Hence, the core purpose of this work is to investigate improving the applicability and usability of BCI systems, whilst preserving signal decoding accuracy. Recent advances in Deep Neural Networks (DNN) provide the possibility for signal processing to automatically learn the best representation of a signal, contributing to improved performance even with a noisy input signal. Subsequently, this thesis focuses on the use of novel DNN-based approaches for tackling some of the key underlying constraints within the area of BCI. For example, recent technological improvements in acquisition hardware have made it possible to eliminate the pre-existing rigid experimentation procedure, albeit resulting in noisier signal capture. However, through the use of a DNN-based model, it is possible to preserve the accuracy of the predictions from the decoded signals. Moreover, this research demonstrates that by leveraging DNN-based image and signal understanding, it is feasible to facilitate real-time BCI applications in a natural environment. Additionally, the capability of DNN to generate realistic synthetic data is shown to be a potential solution in reducing the requirement for costly data collection. Work is also performed in addressing the well-known issues regarding subject bias in BCI models by generating data with reduced subject-specific features. The overall contribution of this thesis is to address the key fundamental limitations of BCI systems. This includes the unyielding traditional experimentation procedure, the mandatory extended calibration stage and sustaining accurate signal decoding in real-time. These limitations lead to a fragile BCI system that is demanding to use and only suited for deployment in a controlled laboratory. Overall contributions of this research aim to improve the robustness of BCI systems and enable new applications for use in the real-world

A Hybrid Brain-Computer Interface for Closed- Loop Position Control of a Robot Arm

Brain-Computer Interfacing has currently added a new dimension in assistive robotics. Existing brain-computer interfaces designed for position control applications suffer from two fundamental limitations. First, most of the existing schemes employ open-loop control, and thus are unable to track the positional errors, resulting in failures in taking necessary online corrective actions. There are traces of one or fewer works dealing with closed-loop EEG-based position control. The existing closed-loop brain-induced position control schemes employ a fixed order link selection rule, which often creates a bottleneck for time-efficient control. Second, the existing brain-induced position controllers are designed to generate the position response like a traditional first-order system, resulting in a large steady-state error. This paper overcomes the above two limitations by keeping provisions for (Steady-State Visual Evoked Potential induced) link-selection in an arbitrary order as required for efficient control and also to generate a second-order response of the position-control system with gradually diminishing overshoots/undershoots to reduce steady-state errors. Besides the above, the third novelty is to utilize motor imagery and P300 signals to design the hybrid brain-computer interfacing system for the said application with gradually diminishing error-margin by speed reversal at the zero-crossings of positional errors. Experiments undertaken reveal that the steady-state error is reduced to 0.2%. The paper also provides a thorough analysis of stability of the closed-loop system performance using Root Locus technique

Advances in Robot Navigation

Robot navigation includes different interrelated activities such as perception - obtaining and interpreting sensory information; exploration - the strategy that guides the robot to select the next direction to go; mapping - the construction of a spatial representation by using the sensory information perceived; localization - the strategy to estimate the robot position within the spatial map; path planning - the strategy to find a path towards a goal location being optimal or not; and path execution, where motor actions are determined and adapted to environmental changes. This book integrates results from the research work of authors all over the world, addressing the abovementioned activities and analyzing the critical implications of dealing with dynamic environments. Different solutions providing adaptive navigation are taken from nature inspiration, and diverse applications are described in the context of an important field of study: social robotics

Machine Vision-based Obstacle Avoidance for Mobile Robot

Obstacle avoidance for mobile robots, especially humanoid robot, is an essential ability for the robot to perform in its environment. This ability based on the colour recognition capability of the barrier or obstacle and the field, as well as the ability to perform movements avoiding the barrier, detected when the robot detects an obstacle in its path. This research develops a detection system of barrier objects and a field with a colour range in HSV format and extracts the edges of barrier objects with the FindContoure method at a threshold filter value. The filter results are then processed using the Bounding Rect method so that the results are obtained from the object detection coordinate extraction. The test results detect the colour of the barrier object with OpenCV is 100%, the movement test uses the processing of the object's colour image and robot direction based on the contour area value&gt; 12500 Pixels, the percentage of the robot making edging motion through the red barrier object is 80% and the contour area testing &lt;12500 pixel is 70% of the movement of the robot forward approaching the barrier object

Smart Sensors for Healthcare and Medical Applications

This book focuses on new sensing technologies, measurement techniques, and their applications in medicine and healthcare. Specifically, the book briefly describes the potential of smart sensors in the aforementioned applications, collecting 24 articles selected and published in the Special Issue “Smart Sensors for Healthcare and Medical Applications”. We proposed this topic, being aware of the pivotal role that smart sensors can play in the improvement of healthcare services in both acute and chronic conditions as well as in prevention for a healthy life and active aging. The articles selected in this book cover a variety of topics related to the design, validation, and application of smart sensors to healthcare

Personality Identification from Social Media Using Deep Learning: A Review

Social media helps in sharing of ideas and information among people scattered around the world and thus helps in creating communities, groups, and virtual networks. Identification of personality is significant in many types of applications such as in detecting the mental state or character of a person, predicting job satisfaction, professional and personal relationship success, in recommendation systems. Personality is also an important factor to determine individual variation in thoughts, feelings, and conduct systems. According to the survey of Global social media research in 2018, approximately 3.196 billion social media users are in worldwide. The numbers are estimated to grow rapidly further with the use of mobile smart devices and advancement in technology. Support vector machine (SVM), Naive Bayes (NB), Multilayer perceptron neural network, and convolutional neural network (CNN) are some of the machine learning techniques used for personality identification in the literature review. This paper presents various studies conducted in identifying the personality of social media users with the help of machine learning approaches and the recent studies that targeted to predict the personality of online social media (OSM) users are reviewed