A System for True and False Memory Prediction Based on 2D and 3D Educational Contents and EEG Brain Signals

We studied the impact of 2D and 3D educational contents on learning and memory recall using electroencephalography (EEG) brain signals. For this purpose, we adopted a classification approach that predicts true and false memories in case of both short term memory (STM) and long term memory (LTM) and helps to decide whether there is a difference between the impact of 2D and 3D educational contents. In this approach, EEG brain signals are converted into topomaps and then discriminative features are extracted from them and finally support vector machine (SVM) which is employed to predict brain states. For data collection, half of sixty-eight healthy individuals watched the learning material in 2D format whereas the rest watched the same material in 3D format. After learning task, memory recall tasks were performed after 30 minutes (STM) and two months (LTM), and EEG signals were recorded. In case of STM, 97.5% prediction accuracy was achieved for 3D and 96.6% for 2D and, in case of LTM, it was 100% for both 2D and 3D. The statistical analysis of the results suggested that for learning and memory recall both 2D and 3D materials do not have much difference in case of STM and LTM

Socio-Cognitive and Affective Computing

Social cognition focuses on how people process, store, and apply information about other people and social situations. It focuses on the role that cognitive processes play in social interactions. On the other hand, the term cognitive computing is generally used to refer to new hardware and/or software that mimics the functioning of the human brain and helps to improve human decision-making. In this sense, it is a type of computing with the goal of discovering more accurate models of how the human brain/mind senses, reasons, and responds to stimuli. Socio-Cognitive Computing should be understood as a set of theoretical interdisciplinary frameworks, methodologies, methods and hardware/software tools to model how the human brain mediates social interactions. In addition, Affective Computing is the study and development of systems and devices that can recognize, interpret, process, and simulate human affects, a fundamental aspect of socio-cognitive neuroscience. It is an interdisciplinary field spanning computer science, electrical engineering, psychology, and cognitive science. Physiological Computing is a category of technology in which electrophysiological data recorded directly from human activity are used to interface with a computing device. This technology becomes even more relevant when computing can be integrated pervasively in everyday life environments. Thus, Socio-Cognitive and Affective Computing systems should be able to adapt their behavior according to the Physiological Computing paradigm. This book integrates proposals from researchers who use signals from the brain and/or body to infer people's intentions and psychological state in smart computing systems. The design of this kind of systems combines knowledge and methods of ubiquitous and pervasive computing, as well as physiological data measurement and processing, with those of socio-cognitive and affective computing

Recent Advances in Signal Processing

The signal processing task is a very critical issue in the majority of new technological inventions and challenges in a variety of applications in both science and engineering fields. Classical signal processing techniques have largely worked with mathematical models that are linear, local, stationary, and Gaussian. They have always favored closed-form tractability over real-world accuracy. These constraints were imposed by the lack of powerful computing tools. During the last few decades, signal processing theories, developments, and applications have matured rapidly and now include tools from many areas of mathematics, computer science, physics, and engineering. This book is targeted primarily toward both students and researchers who want to be exposed to a wide variety of signal processing techniques and algorithms. It includes 27 chapters that can be categorized into five different areas depending on the application at hand. These five categories are ordered to address image processing, speech processing, communication systems, time-series analysis, and educational packages respectively. The book has the advantage of providing a collection of applications that are completely independent and self-contained; thus, the interested reader can choose any chapter and skip to another without losing continuity

The Mental Representation of Visual Information

Despite working in relative independence, the working memory and imagery literatures investigate the mental representation of visual information. Recent reports investigating the neural structure and their associated functional activity responsible for the creation and maintenance of these cognitive representations suggest a significant overlap between these fields of study. Because each field has adopted methodologies that does not allow for a direct comparison of the mental representation described by their respective literatures, it is difficult to determine whether imagery and working memory representations are related. Hence, the current thesis further investigates the properties of the visual representation of visual information to bridge between the imagery and working memory fields. In a first study, I compare the psychophysical properties of simple stimuli commonly used in working memory reports with more complex objects adopted by the imagery field. In the course of three experiments, I demonstrate that the cost of stimulus complexity predominantly affects the quality of the mental representation while still providing evidence of a shared cognitive mechanism driving the formation and maintenance of these representations. In a second study, I evaluate the impact of mental rotation on these mental representations as well as whether the adoption of different paradigms, along with different performance metrics, assess the same cognitive construct. Here again, I show strong evidence in support of a common cognitive mechanism driving the performance across mental manipulation and through assessment methods. Finally, the last study attempted to track the manipulation of these visual representations by applying an encoding model to raw EEG activity. While I show evidence of the orientation-relevant activity during perception, the encoding model does not detect reliable enough activity to allow for tracking the orientation of the stimulus during retention and mental rotation. Together, this thesis provides evidence of a shared cognitive mechanism that drives visual working memory and imagery representation, but tracking these mental representations using EEG activity during manipulation remains unclear

Epileptic seizure detection from electroencephalogram (EEG) signals using linear graph convolutional network and DenseNet based hybrid framework

A clinical condition known as epilepsy occurs when the brain's regular electrical activity is disturbed, resulting in a rapid, aberrant, and excessive discharge of brain neurons. The electroencephalogram (EEG) signal is the measurement of electrical activity received from the nerve cells of the cerebral cortex to make precise diagnoses of disorders, which is made crucial attention for treating epilepsy patients in recent years. The concentration on grid-like data has been a significant drawback of existing deep learning-based automatic epileptic seizure detection algorithms from raw EEG signals; nevertheless, physiological recordings frequently have irregular and unordered structures, making it challenging to think of them as a matrix. In order to take advantage of the implicit information that exists in seizure detection, graph neural networks have received a lot of attention. These networks feature interacting nodes connected by edges whose weights can be either dictated by temporal correlations or anatomical junctions. To address this limitation, a novel hybrid framework is proposed for epileptic seizure detection by using linear graph convolution neural network (LGCN) and DenseNet. When compared to previous deep learning networks, DenseNet achieves the model's higher computational accuracy and memory efficiency by reducing the vanishing gradient problem and enhancing feature propagation in each of its layers. The Stockwell transform (S-transform) is used to preprocess from the raw EEG signal and then group the resulting matrix into time-frequency blocks as inputs for the LGCN to use for feature selection and after the Densenet uses for classification. The proposed hybrid framework outperforms the state-of-the-art in seizure detection tasks, achieving 98% accuracy and 98.60% specificity in extensive experiments on the publicly available CHB-MIT EEG dataset

A framework to measure human behaviour whilst reading

The brain is the most complex object in the known universe that gives a sense of being to humans and characterises human behaviour. Building models of brain functions is perhaps the most fascinating scientific challenge in the 21st century. Reading is a significant cognitive process in the human brain that plays a critical role in the vital process of learning and in performing some daily activities. The study of human behaviour during reading has been an area of interest for researchers in different fields of science. This thesis is based upon providing a novel framework, called ARSAT (Assisting Researchers in the Selection of Appropriate Technologies), that measures the behaviour of humans when reading text. The ARSAT framework aims at assisting researchers in the selection and application of appropriate technologies to measure the behaviour of a person who is reading text. The ARSAT framework will assist to researchers who investigate the reading process and find it difficult to select appropriate theories, metrics, data collection methods and data analytics techniques. The ARSAT framework enhances the ability of its users to select appropriate metrics indicating the effective factors on the characterisation of different aspects of human behaviour during the reading process. As will be shown in this research study, human behaviour is characterised by a complicated interplay of action, cognition and emotion. The ARSAT framework also facilitates selecting appropriate sensory technologies that can be used to monitor and collect data for the metrics. Moreover, this research study will introduce BehaveNet, a novel Deep Learning modelling approach, which can be used for training Deep Learning models of human behaviour from the sensory data collected. In this thesis, a comprehensive literature study is presented that was conducted to acquire adequate knowledge for designing the ARSAT framework. In order to identify the contributing factors that affect the reading process, an overview of some existing theories of the reading process is provided. Furthermore, a number of sensory technologies and techniques that can be applied to monitoring the changes in the metrics indicating the factors are also demonstrated. Only, the technologies that are commercially available on the market are recommended by the ARSAT framework. A variety of Machine Learning techniques were also investigated when designing the BehaveNet. The BehaveNet takes advantage of the complementarity of Convolutional Neural Networks, Long Short-Term Memory networks and Deep Neural Networks. The design of a Human Behaviour Monitoring System (HBMS), by utilising the ARSAT framework for recognising three attention-seeking activities of humans, is also presented in this research study. Reading printed text, as well as speaking out loudly and watching a programme on TV were proposed as activities that a person unintentionally may shift his/her attention from reading into distractions. Between sensory devices recommended by the ARSAT framework, the Muse headband which is an Electroencephalography (EEG) and head motion-sensing wearable device, was selected to track the forehead EEG and a person’s head movements. The EEG and 3-axes accelerometer data were recorded from eight participants when they read printed text, as well as the time they performed two other activities. An imbalanced dataset consisting over 1.2 million rows of noisy data was created and used to build a model of the activities (60% training and 20% validating data) and evaluating the model (20% of the data). The efficiency of the framework is demonstrated by comparing the performance of the models built by utilising the BehaveNet, with the models built by utilising a number of competing Deep Learning models for raw EEG and accelerometer data, that have attained state-of-the-art performance. The classification results are evaluated by some metrics including the classification accuracy, F1 score, confusion matrix, Receiver Operating Characteristic curve, and Area under Curve (AUC) score. By considering the results, the BehaveNet contributed to the body of knowledge as an approach for measuring human behaviour by using sensory devices. In comparison with the performance of the other models, the models built by utilising the BehaveNet, attained better performance when classifying data of two EEG channels (Accuracy = 95%; AUC=0.99; F1 = 0.95), data of a single EEG channel (Accuracy = 85%; AUC=0.96; F1 = 0.83), accelerometer data (Accuracy = 81%; AUC = 0.9; F1 = 0.76) and all of the data in the dataset (Accuracy = 97%; AUC = 0.99; F1 = 0.96). The dataset and the source code of this project are also published on the Internet to help the science community. The Muse headband is also shown to be an economical and standard wearable device that can be successfully used in behavioural research