Affective e-learning approaches, technology and implementation model: a systematic review

A systematic literature study including articles from 2016 to 2022 was done to evaluate the various approaches, technologies, and implementation models involved in measuring student engagement during learning. The review’s objective was to compile and analyze all studies that investigated how instructors can gauge students’ mental states while teaching and assess the most effective teaching methods. Additionally, it aims to extract and assess expanded methodologies from chosen research publications to offer suggestions and answers to researchers and practitioners. Planning, carrying out the analysis, and publishing the results have all received significant attention in the research approach. The study’s findings indicate that more needs to be done to evaluate student participation objectively and follow their development for improved academic performance. Physiological approaches should be given more support among the alternatives. While deep learning implementation models and contactless technology should interest more researchers. And, the recommender system should be integrated into e-learning system. Other approaches, technologies, and methodology articles, on the other hand, lacked authenticity in conveying student feeling

Affect-driven Engagement Measurement from Videos

In education and intervention programs, person's engagement has been identified as a major factor in successful program completion. Automatic measurement of person's engagement provides useful information for instructors to meet program objectives and individualize program delivery. In this paper, we present a novel approach for video-based engagement measurement in virtual learning programs. We propose to use affect states, continuous values of valence and arousal extracted from consecutive video frames, along with a new latent affective feature vector and behavioral features for engagement measurement. Deep learning-based temporal, and traditional machine-learning-based non-temporal models are trained and validated on frame-level, and video-level features, respectively. In addition to the conventional centralized learning, we also implement the proposed method in a decentralized federated learning setting and study the effect of model personalization in engagement measurement. We evaluated the performance of the proposed method on the only two publicly available video engagement measurement datasets, DAiSEE and EmotiW, containing videos of students in online learning programs. Our experiments show a state-of-the-art engagement level classification accuracy of 63.3% and correctly classifying disengagement videos in the DAiSEE dataset and a regression mean squared error of 0.0673 on the EmotiW dataset. Our ablation study shows the effectiveness of incorporating affect states in engagement measurement. We interpret the findings from the experimental results based on psychology concepts in the field of engagement.Comment: 13 pages, 8 figures, 7 table

Body gestures recognition for human robot interaction

In this project, a solution for human gesture classification is proposed. The solution uses a Deep Learning model and is meant to be useful for non-verbal communication between humans and robots. The State-of-the-Art is researched in an effort to achieve a model ready to work with natural gestures without restrictions. The research will focus on the creation of a temPoral bOdy geSTUre REcognition model (POSTURE) that can recognise continuous gestures performed in real-life situations. The suggested model takes into account spatial and temporal components so as to achieve the recognition of more natural and intuitive gestures. In a first step, a framework extracts from all the images the corresponding landmarks for each of the body joints. Next, some data filtering techniques are applied with the aim of avoiding problems related with the data. Afterwards, the filtered data is input into an State-of-the-Art Neural Network. And finally, different neural network configurations and approaches are tested to find the optimal performance. The obtained outcome shows the research has been done in the right track and how, despite of the dataset problems found, even better results can be achievedObjectius de Desenvolupament Sostenible::9 - Indústria, Innovació i Infraestructur

A Survey of Multimedia Technologies and Robust Algorithms

Multimedia technologies are now more practical and deployable in real life, and the algorithms are widely used in various researching areas such as deep learning, signal processing, haptics, computer vision, robotics, and medical multimedia processing. This survey provides an overview of multimedia technologies and robust algorithms in multimedia data processing, medical multimedia processing, human facial expression tracking and pose recognition, and multimedia in education and training. This survey will also analyze and propose a future research direction based on the overview of current robust algorithms and multimedia technologies. We want to thank the research and previous work done by the Multimedia Research Centre (MRC), the University of Alberta, which is the inspiration and starting point for future research.Comment: arXiv admin note: text overlap with arXiv:2010.1296

Ethics and Responsible AI Deployment

As Artificial Intelligence (AI) becomes more prevalent, protecting personal privacy is a critical ethical issue that must be addressed. This article explores the need for ethical AI systems that safeguard individual privacy while complying with ethical standards. By taking a multidisciplinary approach, the research examines innovative algorithmic techniques such as differential privacy, homomorphic encryption, federated learning, international regulatory frameworks, and ethical guidelines. The study concludes that these algorithms effectively enhance privacy protection while balancing the utility of AI with the need to protect personal data. The article emphasises the importance of a comprehensive approach that combines technological innovation with ethical and regulatory strategies to harness the power of AI in a way that respects and protects individual privacy

From Short-Term Tolerance to Long-Term Recognition in Human Visual Memory

Humans have a remarkable ability to recognize visual objects following limited exposure and despite changes at the image-level. How humans acquire this ability remains a mystery, and it remains one area in which artificial intelligence has yet to match human performance. I sought to understand this fundamental cognitive ability by leveraging theories and methods from multiple fields. In particular, I examined how rules guiding the perception of objects in visual working memory assist in the construction of visual long-term memories. In four experiments, I reveal that our expectations for how objects move in the world are used to learn and integrate object information into visual long-term memory. Next, I further examined how aspects of memory over the short-term may actually be features used to construct appropriately constrained representations in the long-term. I demonstrate in three experiments that visual working memory is highly tolerant to variability at test, in order to act as a venue to integrate information into long-term memory. Finally, I moved past investigating memory following singular experiences to understand how our memories change over repeated exposures. I discovered across three experiments that the initial quality of an experience, as well the amount of time between repeated encounters, affects our ability to integrate and remember objects we encounter multiple times. This work contributes to our understanding of the growth process of visual memory, and attempts to form bridges between traditionally disparate fields of vision scientist studying object perception, neuroscientists studying long-term memory, and engineers designing artificial intelligence recognition systems

Decoding non-invasive brain activity with novel deep-learning approaches

This thesis delves into the world of non-invasive electrophysiological brain signals like electroencephalography (EEG) and magnetoencephalography (MEG), focusing on modelling and decoding such data. The research aims to investigate what happens in the brain when we perceive visual stimuli or engage in covert speech (inner speech) and enhance the decoding performance of such stimuli. The findings have significant implications for the development of brain-computer interfaces (BCIs), leading to assistive communication technologies for paralysed individuals. The thesis is divided into two main sections, methodological and experimental work. A central concern in both sections is the large variability present in electrophysiological recordings, whether it be within-subject or between-subject variability, and to a certain extent between-dataset variability. In the methodological sections, we explore the potential of deep learning for brain decoding. The research acknowledges the urgent need for more sophisticated models and larger datasets to improve the decoding and modelling of EEG and MEG signals. We present advancements in decoding visual stimuli using linear models at the individual subject level. We then explore how deep learning techniques can be employed for group decoding, introducing new methods to deal with between-subject variability. Finally, we also explores novel forecasting models of MEG data based on convolutional and Transformer-based architectures. In particular, Transformer-based models demonstrate superior capabilities in generating signals that closely match real brain data, thereby enhancing the accuracy and reliability of modelling the brain’s electrophysiology. In the experimental section, we present a unique dataset containing high-trial inner speech EEG, MEG, and preliminary optically pumped magnetometer (OPM) data. We highlight the limitations of current BCI systems used for communication, which are either invasive or extremely slow. While inner speech decoding from non-invasive brain signals has great promise, it has been a challenging goal in the field with limited decoding approaches, indicating a significant gap that needs to be addressed. Our aim is to investigate different types of inner speech and push decoding performance by collecting a high number of trials and sessions from a few participants. However, the decoding results are found to be mostly negative, underscoring the difficulty of decoding inner speech. In conclusion, this thesis provides valuable insight into the challenges and potential solutions in the field of electrophysiology, particularly in the decoding of visual stimuli and inner speech. The findings could pave the way for future research and advancements in the field, ultimately improving communication capabilities for paralysed individuals