An Ordinal Approach to Affective Computing

Both depression prediction and emotion recognition systems are often based on ordinal ground truth due to subjectively annotated datasets. Yet, both have so far been posed as classification or regression problems. These naive approaches have fundamental issues because they are not focused on ordering, unlike ordinal regression, which is the most appropriate for truly ordinal ground truth. Ordinal regression to date offers comparatively fewer, more limited methods when compared with other branches in machine learning, and its usage has been limited to specific research domains. Accordingly, this thesis presents investigations into ordinal approaches for affective computing by describing a consistent framework to understand all ordinal system designs, proposing ordinal systems for large datasets, and introducing tools and principles to select suitable system designs and evaluation methods. First, three learning approaches are compared using the support vector framework to establish the empirical advantages of ordinal regression, which is lacking from the current literature. Results on depression and emotion corpora indicate that ordinal regression with proper tuning can improve existing depression and emotion systems. Ordinal logistic regression (OLR), which is an extension of logistic regression for ordinal scales, contributes to a number of model structures, from which the best structure must be chosen. Exploiting the newly proposed computationally efficient greedy algorithm for model structure selection (GREP), OLR outperformed or was comparable with state-of-the-art depression systems on two benchmark depression speech datasets. Deep learning has dominated many affective computing fields, and hence ordinal deep learning is an attractive prospect. However, it is under-studied even in the machine learning literature, which motivates an in-depth analysis of appropriate network architectures and loss functions. One of the significant outcomes of this analysis is the introduction of RankCNet, a novel ordinal network which utilises a surrogate loss function of rank correlation. Not only the modelling algorithm but the choice of evaluation measure depends on the nature of the ground truth. Rank correlation measures, which are sensitive to ordering, are more apt for ordinal problems than common classification or regression measures that ignore ordering information. Although rank-based evaluation for ordinal problems is not new, so far in affective computing, ordinality of the ground truth has been widely ignored during evaluation. Hence, a systematic analysis in the affective computing context is presented, to provide clarity and encourage careful choice of evaluation measures. Another contribution is a neural network framework with a novel multi-term loss function to assess the ordinality of ordinally-annotated datasets, which can guide the selection of suitable learning and evaluation methods. Experiments on multiple synthetic and affective speech datasets reveal that the proposed system can offer reliable and meaningful predictions about the ordinality of a given dataset. Overall, the novel contributions and findings presented in this thesis not only improve prediction accuracy but also encourage future research towards ordinal affective computing: a different paradigm, but often the most appropriate

Classifying depression symptom severity: Assessment of speech representations in personalized and generalized machine learning models

There is an urgent need for new methods that improve the management and treatment of Major Depressive Disorder (MDD). Speech has long been regarded as a promising digital marker in this regard, with many works highlighting that speech changes associated with MDD can be captured through machine learning models. Typically, findings are based on cross-sectional data, with little work exploring the advantages of personalization in building more robust and reliable models. This work assesses the strengths of different combinations of speech representations and machine learning models, in personalized and generalized settings in a two-class depression severity classification paradigm. Key results on a longitudinal dataset highlight the benefits of personalization. Our strongest performing model set-up utilized self-supervised learning features and convolutional neural network (CNN) and long short-term memory (LSTM) back-end

Deep Multi-task Learning for Depression Detection and Prediction in Longitudinal Data

Depression is among the most prevalent mental disorders, affecting millions of people of all ages globally. Machine learning techniques have shown effective in enabling automated detection and prediction of depression for early intervention and treatment. However, they are challenged by the relative scarcity of instances of depression in the data. In this work we introduce a novel deep multi-task recurrent neural network to tackle this challenge, in which depression classification is jointly optimized with two auxiliary tasks, namely one-class metric learning and anomaly ranking. The auxiliary tasks introduce an inductive bias that improves the classification model's generalizability on small depression samples. Further, unlike existing studies that focus on learning depression signs from static data without considering temporal dynamics, we focus on longitudinal data because i) temporal changes in personal development and family environment can provide critical cues for psychiatric disorders and ii) it may enable us to predict depression before the illness actually occurs. Extensive experimental results on child depression data show that our model is able to i) achieve nearly perfect performance in depression detection and ii) accurately predict depression 2-4 years before the clinical diagnosis, substantially outperforming seven competing methods.Comment: 9 pages, 3 figures, 3 table

Novel Machine Learning and Wearable Sensor Based Solutions for Smart Healthcare Monitoring

The advent of IoT has enabled the design of connected and integrated smart health monitoring systems. These health monitoring systems can be utilized for monitoring the mental and physical wellbeing of a person. Stress, anxiety, and hypertension are the major elements responsible for the plethora of physical and mental illnesses. In this context, the older population demands special attention because of the several age-related complications that exacerbate the effects of stress, anxiety, and hypertension. Monitoring stress, anxiety, and blood pressure regularly can prevent long-term damage by initiating necessary intervention or clinical treatment beforehand. This will improve the quality of life and reduce the burden on caregivers and the cost of healthcare. Therefore, this thesis explores novel technological solutions for real-time monitoring of stress, anxiety, and blood pressure using unobtrusive wearable sensors and machine learning techniques. The first contribution of this thesis is the experimental data collection of 50 healthy older adults, based on which, the works on stress detection and anxiety detection have been developed. The data collection procedure lasted for more than a year. We have collected physiological signals, salivary cortisol, and self-reported questionnaire feedback during the study. Salivary cortisol is an established clinical biomarker for physiological stress. Hence, a stress detection model that is trained to distinguish between the stressed and not-stressed states as indicated by the increase in cortisol level has the potential to facilitate clinical level diagnosis of stress from the comfort of their own home. The second contribution of the thesis is the development of a stress detection model based on fingertip sensors. We have extracted features from Electrodermal Activity (EDA) and Blood Volume Pulse (BVP) signals obtained from fingertip EDA and Photoplethysmogram (PPG) sensors to train machine learning algorithms for distinguishing between stressed and not-stressed states. We have evaluated the performance of four traditional machine learning algorithms and one deep-learning-based Long Short-Term Memory (LSTM) classifier. Results and analysis showed that the proposed LSTM classifier performed equally well as the traditional machine learning models. The third contribution of the thesis is to evaluate an integrated system of wrist-worn sensors for stress detection. We have evaluated four signal streams, EDA, BVP, Inter-Beat Interval (IBI), and Skin Temperature (ST) signals from EDA, PPG, and ST sensors. A random forest classifier was used for distinguishing between the stressed and not-stressed states. Results and analysis showed that incorporating features from different signals was able to reduce the misclassification rate of the classifier. Further, we have also prototyped the integration of the proposed wristband-based stress detection system in a consumer end device with voice capabilities. The fourth contribution of the thesis is the design of an anxiety detection model that uses features from a single wearable sensor and a context feature to improve the performance of the classification model. Using a context feature instead of integrating other physiological features for improving the performance of the model can reduce the complexity and cost of the anxiety detection model. In our proposed work, we have used a simple experimental context feature to highlight the importance of context in the accurate detection of anxious states. Our results and analysis have shown that with the addition of the context-based feature, the classifier was able to reduce misclassification by increasing the confidence of the decision. The final and the fifth contribution of the thesis is the validation of a proposed computational framework for the blood pressure estimation model. The proposed framework uses features from the PPG signal to estimate the systolic and diastolic blood pressure values using advanced regression techniques

First impressions: A survey on vision-based apparent personality trait analysis

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Personality analysis has been widely studied in psychology, neuropsychology, and signal processing fields, among others. From the past few years, it also became an attractive research area in visual computing. From the computational point of view, by far speech and text have been the most considered cues of information for analyzing personality. However, recently there has been an increasing interest from the computer vision community in analyzing personality from visual data. Recent computer vision approaches are able to accurately analyze human faces, body postures and behaviors, and use these information to infer apparent personality traits. Because of the overwhelming research interest in this topic, and of the potential impact that this sort of methods could have in society, we present in this paper an up-to-date review of existing vision-based approaches for apparent personality trait recognition. We describe seminal and cutting edge works on the subject, discussing and comparing their distinctive features and limitations. Future venues of research in the field are identified and discussed. Furthermore, aspects on the subjectivity in data labeling/evaluation, as well as current datasets and challenges organized to push the research on the field are reviewed.Peer ReviewedPostprint (author's final draft

Robust subspace learning for static and dynamic affect and behaviour modelling

Machine analysis of human affect and behavior in naturalistic contexts has witnessed a growing attention in the last decade from various disciplines ranging from social and cognitive sciences to machine learning and computer vision. Endowing machines with the ability to seamlessly detect, analyze, model, predict as well as simulate and synthesize manifestations of internal emotional and behavioral states in real-world data is deemed essential for the deployment of next-generation, emotionally- and socially-competent human-centered interfaces. In this thesis, we are primarily motivated by the problem of modeling, recognizing and predicting spontaneous expressions of non-verbal human affect and behavior manifested through either low-level facial attributes in static images or high-level semantic events in image sequences. Both visual data and annotations of naturalistic affect and behavior naturally contain noisy measurements of unbounded magnitude at random locations, commonly referred to as ‘outliers’. We present here machine learning methods that are robust to such gross, sparse noise. First, we deal with static analysis of face images, viewing the latter as a superposition of mutually-incoherent, low-complexity components corresponding to facial attributes, such as facial identity, expressions and activation of atomic facial muscle actions. We develop a robust, discriminant dictionary learning framework to extract these components from grossly corrupted training data and combine it with sparse representation to recognize the associated attributes. We demonstrate that our framework can jointly address interrelated classification tasks such as face and facial expression recognition. Inspired by the well-documented importance of the temporal aspect in perceiving affect and behavior, we direct the bulk of our research efforts into continuous-time modeling of dimensional affect and social behavior. Having identified a gap in the literature which is the lack of data containing annotations of social attitudes in continuous time and scale, we first curate a new audio-visual database of multi-party conversations from political debates annotated frame-by-frame in terms of real-valued conflict intensity and use it to conduct the first study on continuous-time conflict intensity estimation. Our experimental findings corroborate previous evidence indicating the inability of existing classifiers in capturing the hidden temporal structures of affective and behavioral displays. We present here a novel dynamic behavior analysis framework which models temporal dynamics in an explicit way, based on the natural assumption that continuous- time annotations of smoothly-varying affect or behavior can be viewed as outputs of a low-complexity linear dynamical system when behavioral cues (features) act as system inputs. A novel robust structured rank minimization framework is proposed to estimate the system parameters in the presence of gross corruptions and partially missing data. Experiments on prediction of dimensional conflict and affect as well as multi-object tracking from detection validate the effectiveness of our predictive framework and demonstrate that for the first time that complex human behavior and affect can be learned and predicted based on small training sets of person(s)-specific observations.Open Acces

The measurement of optimism and hope in relation to college student retention and academic success

Consistent with research approaches in positive psychology, the variables of optimism and hope were investigated to determine what role these constructs had in the retention and academic success of college students. A sample of 282 students (172 women, 111 men) from a large midwestern university, supplemented by 82 collateral reporters (friends or relatives of the students), participated in the study. Each participant completed a questionnaire including Scheier and Carvers\u27 Life Orientation Test (LOT); Peterson, Semrnel, von Baeyer, Abramson, Metalsky, and Seligmans\u27 Attributional Style Questionnaire (ASQ), and C. R. Snyder\u27s Adult Trait Hope Scale (HOPE) along with measures of depression, anxiety, commitment to college, and academic motivation. Participants gave consent for the release of their college grade point average, ACT composite score, and high school class rank from the registrar\u27s office. In addition, the collateral reporters were asked to complete the LOT, ASQ, and HOPE according to their knowledge of the participants.;Analyses including linear and logistic regression and Pearson correlations provided evidence for a significant relationship between hope and grade point average, but not optimism. Additionally, no significant relationships were found between optimism, hope, and continued enrollment from the fall to spring semester. Data from the collateral reporters demonstrated a similar pattern with hope significantly related to grade point average as indicated by correlation and regression analyses. The need for further clarification of the relationship between hope, academic success, and retention was discussed along with the limitations of the current study and suggestions for additional research

Predicting the Risk of Falling with Artificial Intelligence

Predicting the Risk of Falling with Artificial Intelligence Abstract Background: Fall prevention is a huge patient safety concern among all healthcare organizations. The high prevalence of patient falls has grave consequences, including the cost of care, longer hospital stays, unintentional injuries, and decreased patient and staff satisfaction. Preventing a patient from falling is critical in maintaining a patient’s quality of life and averting the high cost of healthcare expenses. Local Problem: Two hospitals\u27 healthcare system saw a significant increase in inpatient falls. The fall rate is one of the nursing quality indicators, and fall reduction is a key performance indicator of high-quality patient care. Methods: This quality improvement evidence-based observational project compared the rate of fall (ROF) between the experimental and control unit. Pearson’s chi-square and Fisher’s exact test were used to analyze and compare results. Qualtrics surveys evaluated the nurses’ perception of AI, and results were analyzed using the Mann-Whitney Rank Sum test. Intervention. Implementing an artificial intelligence-assisted fall predictive analytics model that can timely and accurately predict fall risk can mitigate the increase in inpatient falls. Results: The pilot unit (Pearson’s chi-square = p pp\u3c0.001). Conclusions: AI-assisted automatic fall predictive risk assessment produced a significant reduction if the number of falls, the ROF, and the use of fall countermeasures. Further, nurses’ perception of AI improved after the introduction of FPAT and presentation