Robust Modeling of Epistemic Mental States

This work identifies and advances some research challenges in the analysis of facial features and their temporal dynamics with epistemic mental states in dyadic conversations. Epistemic states are: Agreement, Concentration, Thoughtful, Certain, and Interest. In this paper, we perform a number of statistical analyses and simulations to identify the relationship between facial features and epistemic states. Non-linear relations are found to be more prevalent, while temporal features derived from original facial features have demonstrated a strong correlation with intensity changes. Then, we propose a novel prediction framework that takes facial features and their nonlinear relation scores as input and predict different epistemic states in videos. The prediction of epistemic states is boosted when the classification of emotion changing regions such as rising, falling, or steady-state are incorporated with the temporal features. The proposed predictive models can predict the epistemic states with significantly improved accuracy: correlation coefficient (CoERR) for Agreement is 0.827, for Concentration 0.901, for Thoughtful 0.794, for Certain 0.854, and for Interest 0.913.Comment: Accepted for Publication in Multimedia Tools and Application, Special Issue: Socio-Affective Technologie

EDIBLE FISH IDENTIFICATION BASED ON MACHINE LEARNING

Automated fish identification system has a beneficial role in various fields. Fish species can usually be identified based on visual observation and human experiences. False appreciation can cause food poisoning. The proposed system aims to efficiently and effectively identify edible fish from poisonous ones based on three machine learning (ML) techniques. A total of 300 fish images are used, collected from 20 species with differences in shapes, sizes, and colors. Hybrid features were extracted and then fed to three types of ML techniques: k-nearest neighbor (K-NN), support vector machine (SVM), and neural networks (NN). The 300 fish images are divided into two: 70% for training and 30% for testing. The accuracy rates for the presented system were 91.1%, 92.2%, and 94.4% for KNN, SVM, and NNs, respectively. The proposed system is evaluated using four terms: precision, sensitivity, F1-score, and accuracy. Results show that the proposed approach achieved higher accuracy compared with other recent pertinent studies

CHEATING DETECTION IN ONLINE EXAMS BASED ON CAPTURED VIDEO USING DEEP LEARNING

Today, e-learning has become a reality and a global trend imposed and accelerated by the COVID-19 pandemic. However, there are many risks and challenges related to the credibility of online exams which are of widespread concern to educational institutions around the world. Online exam system continues to gain popularity, particularly during the pandemic, due to the rapid expansion of digitalization and globalization. To protect the integrity of the examination and provide objective and fair results, cheating detection and prevention in examination systems is a must. Therefore, the main objective of this thesis is to develop an effective way of detection of cheating in online exams. In this work, a system to track and prevent attempts to cheat on online exams is developed using artificial intelligence techniques. The suggested solution uses the webcam that is already connected to the computer to record videos of the examinee in real time and afterwards analyze them using different deep learning methods to find best combinations of models for face detection and classification if cheating/not cheating occurred. To evaluate the system, we use a benchmark dataset of exam videos from 24 participants who represented examinees in online exam. An object detection technique is used to detect face appeared in the image and crop the face portion, and then a deep learning based classification model is trained from the images to classify a face as cheating or not cheating. We have proposed an effective combination of data preprocessing, object detection, and classification models to obtain high detection accuracy. We believe that the suggested invigilation methodology can be used in colleges, institutions, and schools to look for and keep an eye on suspicious student behavior. Hopefully, by putting the proposed invigilation method into place, we can aid in eliminating and reducing cheating incidences as it undermines the integrity and fairness of the educational system

DeepTMH: Multimodal Semi-supervised framework leveraging Affective and Cognitive engagement for Telemental Health

To aid existing telemental health services, we propose DeepTMH, a novel framework that models telemental health session videos by extracting latent vectors corresponding to Affective and Cognitive features frequently used in psychology literature. Our approach leverages advances in semi-supervised learning to tackle the data scarcity in the telemental health session video domain and consists of a multimodal semi-supervised GAN to detect important mental health indicators during telemental health sessions. We demonstrate the usefulness of our framework and contrast against existing works in two tasks: Engagement regression and Valence-Arousal regression, both of which are important to psychologists during a telemental health session. Our framework reports 40% improvement in RMSE over SOTA method in Engagement Regression and 50% improvement in RMSE over SOTA method in Valence-Arousal Regression. To tackle the scarcity of publicly available datasets in telemental health space, we release a new dataset, MEDICA, for mental health patient engagement detection. Our dataset, MEDICA consists of 1299 videos, each 3 seconds long. To the best of our knowledge, our approach is the first method to model telemental health session data based on psychology-driven Affective and Cognitive features, which also accounts for data sparsity by leveraging a semi-supervised setup