Affective Computing in the Area of Autism

The prevalence rate of Autism Spectrum Disorders (ASD) is increasing at an alarming rate (1 in 68 children). With this increase comes the need of early diagnosis of ASD, timely intervention, and understanding the conditions that could be comorbid to ASD. Understanding co-morbid anxiety and its interaction with emotion comprehension and production in ASD is a growing and multifaceted area of research. Recognizing and producing contingent emotional expressions is a complex task, which is even more difficult for individuals with ASD. First, I investigate the arousal experienced by adolescents with ASD in a group therapy setting. In this study I identify the instances in which the physiological arousal is experienced by adolescents with ASD ( have-it ), see if the facial expressions of these adolescents indicate their arousal ( show-it ), and determine if the adolescents are self-aware of this arousal or not ( know-it ). In order to establish a relationship across these three components of emotion expression and recognition, a multi-modal approach for data collection is utilized. Machine learning techniques are used to determine whether still video images of facial expressions could be used to predict Electrodermal Activity (EDA) data. Implications for the understanding of emotion and social communication difficulties in ASD, as well as future targets for intervention, are discussed. Second, it is hypothesized that a well-designed intervention technique helps in the overall development of children with ASD by improving their level of functioning. I designed and validated a mobile-based intervention designed for teaching social skills to children with ASD. I also evaluated the social skill intervention. Last, I present the research goals behind an mHealth-based screening tool for early diagnosis of ASD in toddlers. The design purpose of this tool is to help people from low-income group, who have limited access to resources. This goal is achieved without burdening the physicians, their staff, and the insurance companies

Reconhecimento de padrões em expressões faciais : algoritmos e aplicações

Orientador: Hélio PedriniTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: O reconhecimento de emoções tem-se tornado um tópico relevante de pesquisa pela comunidade científica, uma vez que desempenha um papel essencial na melhoria contínua dos sistemas de interação humano-computador. Ele pode ser aplicado em diversas áreas, tais como medicina, entretenimento, vigilância, biometria, educação, redes sociais e computação afetiva. Há alguns desafios em aberto relacionados ao desenvolvimento de sistemas emocionais baseados em expressões faciais, como dados que refletem emoções mais espontâneas e cenários reais. Nesta tese de doutorado, apresentamos diferentes metodologias para o desenvolvimento de sistemas de reconhecimento de emoções baseado em expressões faciais, bem como sua aplicabilidade na resolução de outros problemas semelhantes. A primeira metodologia é apresentada para o reconhecimento de emoções em expressões faciais ocluídas baseada no Histograma da Transformada Census (CENTRIST). Expressões faciais ocluídas são reconstruídas usando a Análise Robusta de Componentes Principais (RPCA). A extração de características das expressões faciais é realizada pelo CENTRIST, bem como pelos Padrões Binários Locais (LBP), pela Codificação Local do Gradiente (LGC) e por uma extensão do LGC. O espaço de características gerado é reduzido aplicando-se a Análise de Componentes Principais (PCA) e a Análise Discriminante Linear (LDA). Os algoritmos K-Vizinhos mais Próximos (KNN) e Máquinas de Vetores de Suporte (SVM) são usados para classificação. O método alcançou taxas de acerto competitivas para expressões faciais ocluídas e não ocluídas. A segunda é proposta para o reconhecimento dinâmico de expressões faciais baseado em Ritmos Visuais (VR) e Imagens da História do Movimento (MHI), de modo que uma fusão de ambos descritores codifique informações de aparência, forma e movimento dos vídeos. Para extração das características, o Descritor Local de Weber (WLD), o CENTRIST, o Histograma de Gradientes Orientados (HOG) e a Matriz de Coocorrência em Nível de Cinza (GLCM) são empregados. A abordagem apresenta uma nova proposta para o reconhecimento dinâmico de expressões faciais e uma análise da relevância das partes faciais. A terceira é um método eficaz apresentado para o reconhecimento de emoções audiovisuais com base na fala e nas expressões faciais. A metodologia envolve uma rede neural híbrida para extrair características visuais e de áudio dos vídeos. Para extração de áudio, uma Rede Neural Convolucional (CNN) baseada no log-espectrograma de Mel é usada, enquanto uma CNN construída sobre a Transformada de Census é empregada para a extração das características visuais. Os atributos audiovisuais são reduzidos por PCA e LDA, então classificados por KNN, SVM, Regressão Logística (LR) e Gaussian Naïve Bayes (GNB). A abordagem obteve taxas de reconhecimento competitivas, especialmente em dados espontâneos. A penúltima investiga o problema de detectar a síndrome de Down a partir de fotografias. Um descritor geométrico é proposto para extrair características faciais. Experimentos realizados em uma base de dados pública mostram a eficácia da metodologia desenvolvida. A última metodologia trata do reconhecimento de síndromes genéticas em fotografias. O método visa extrair atributos faciais usando características de uma rede neural profunda e medidas antropométricas. Experimentos são realizados em uma base de dados pública, alcançando taxas de reconhecimento competitivasAbstract: Emotion recognition has become a relevant research topic by the scientific community, since it plays an essential role in the continuous improvement of human-computer interaction systems. It can be applied in various areas, for instance, medicine, entertainment, surveillance, biometrics, education, social networks, and affective computing. There are some open challenges related to the development of emotion systems based on facial expressions, such as data that reflect more spontaneous emotions and real scenarios. In this doctoral dissertation, we propose different methodologies to the development of emotion recognition systems based on facial expressions, as well as their applicability in the development of other similar problems. The first is an emotion recognition methodology for occluded facial expressions based on the Census Transform Histogram (CENTRIST). Occluded facial expressions are reconstructed using an algorithm based on Robust Principal Component Analysis (RPCA). Extraction of facial expression features is then performed by CENTRIST, as well as Local Binary Patterns (LBP), Local Gradient Coding (LGC), and an LGC extension. The generated feature space is reduced by applying Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). K-Nearest Neighbor (KNN) and Support Vector Machine (SVM) algorithms are used for classification. This method reached competitive accuracy rates for occluded and non-occluded facial expressions. The second proposes a dynamic facial expression recognition based on Visual Rhythms (VR) and Motion History Images (MHI), such that a fusion of both encodes appearance, shape, and motion information of the video sequences. For feature extraction, Weber Local Descriptor (WLD), CENTRIST, Histogram of Oriented Gradients (HOG), and Gray-Level Co-occurrence Matrix (GLCM) are employed. This approach shows a new direction for performing dynamic facial expression recognition, and an analysis of the relevance of facial parts. The third is an effective method for audio-visual emotion recognition based on speech and facial expressions. The methodology involves a hybrid neural network to extract audio and visual features from videos. For audio extraction, a Convolutional Neural Network (CNN) based on log Mel-spectrogram is used, whereas a CNN built on Census Transform is employed for visual extraction. The audio and visual features are reduced by PCA and LDA, and classified through KNN, SVM, Logistic Regression (LR), and Gaussian Naïve Bayes (GNB). This approach achieves competitive recognition rates, especially in a spontaneous data set. The second last investigates the problem of detecting Down syndrome from photographs. A geometric descriptor is proposed to extract facial features. Experiments performed on a public data set show the effectiveness of the developed methodology. The last methodology is about recognizing genetic disorders in photos. This method focuses on extracting facial features using deep features and anthropometric measurements. Experiments are conducted on a public data set, achieving competitive recognition ratesDoutoradoCiência da ComputaçãoDoutora em Ciência da Computação140532/2019-6CNPQCAPE

Advanced Augmentative and Alternative Communication System Based in Physiological Control

Dyskinetic Cerebral Palsy (DCP) is mainly characterized by alterations in muscle tone and involuntary movements. Therefore, these people present with difficulties in coordination and movement control, which makes walking difficult and affects their posture when seated. Additionally, their cognitive performance varies between being completely normal and severe mental retardation. People with DCP were selected as the objective of this thesis due to their multiple and complex limitations (speech problems and motor control) and because their capabilities have a great margin for improvement thanks to physiological control systems. Given their communication difficulties, some people with DCP have good motor con-trol and can communicate with written language. However, most have difficulty using Augmentative and Alternative Communication (AAC) systems. People with DCP gen-erally use concept boards to indicate the idea they want to communicate. However, most communication solutions available today are based on proprietary software that makes it difficult to customize the concept board and this type of control system. This is the motivation behind this thesis, with the aim of creating an interface with characteristics, able to be adapted to the user needs and limitations. Thus, this thesis proposes an Augmentative and Alternative Communication System for people with DCP based on physiological control. In addition, an innovative system for direct con-trol of concept boards with EMG is proposed. This control system is based on a physi-cal model that reproduces the muscular mechanical response (stiffness, inertia and viscosity). It allows for a selection of elements thanks to small pulses of EMG signal with sensors on a muscle with motor control. Its main advantage is the possibility of correcting errors during selection associated with uncontrolled muscle impulses, avoid-ing sustained muscle effort and thus reduced fatigue.La Parálisis Cerebral de tipo Discinésica (DCP) se caracteriza principalmente por las alteraciones del tono muscular y los movimientos involuntarios. Por ello, estos pacientes presentan dificultades en la coordinación y en el control de movimientos, lo cual les dificulta el caminar y afecta su postura cuando están sentados. Cabe resaltar que la capacidad cognitiva de las personas con DCP puede variar desde completamente normal, hasta un retraso mental severo. Las personas con DCP han sido seleccionadas como objetivo de esta tesis ya el margen de mejora de sus capacidades es amplio gracias a sistemas de control fisiológico, debido a sus múltiples y complejas limitaciones (problemas de habla y control motor). Debido a sus dificultades de comunicación, algunas personas con DCP se pueden comunicar con lenguaje escrito, siempre y cuando tenga un buen control motor. Sin embargo, la mayoría tienen dificultades para usar sistemas de Comunicación Aumentativos y Alternativos (AAC). De hecho, las personas con DCP utilizan generalmente tableros de conceptos para indicar la idea que quieren transmitir. Sin embargo, la mayoría las soluciones de comunicación disponibles en la actualidad están basadas en software propietario que hacen difícil la personalización del tablero de conceptos y el tipo de sistema de control. Es aquí donde surge esta tesis, con el objetivo de crear una interfaz con esas características, capaz de adaptarse a las necesidades y limitaciones del usuario. De esta forma, esta tesis propone un sistema de comunicación aumentativo y alternativo para personas con DCP basado en control fisiológico. Además, se propone un Sistema innovador de control directo sobre tableros de conceptos basado en EMG. Este Sistema de control se basa en un modelo físico que reproduce la respuesta mecánica muscular (basado en parámetros como Rigidez, Inercia y Viscosidad), permitiendo la selección de elementos gracias a pequeños pulsos de señal EMG con sensores sobre un músculo con control motor. Sus principales ventajas son la posibilidad de corregir errores durante la selección asociado a los impulsos musculares no controlados, evitar el esfuerzo muscular mantenido para alcanzar un nivel y reducir la fatiga.La Paràlisi Cerebral de tipus Discinèsica (DCP) es caracteritza principalment per les alteracions del to muscular i els moviments involuntaris. Per açò, aquests pacients presenten dificultats en la coordinació i en el control de moviments, la qual cosa els dificulta el caminar i afecta la seua postura quan estan asseguts. Cal ressaltar que la capacitat cognitiva de les persones amb DCP pot variar des de completament normal, fins a un retard mental sever. Les persones amb DCP han sigut seleccionades com a objectiu d'aquesta tesi ja el marge de millora de les seues capacitats és ampli gràcies a sistemes de control fisiològic, a causa dels seus múltiples i complexes limitacions (problemes de parla i control motor). A causa de les seues dificultats de comunicació, algunes persones amb DCP es poden comunicar amb llenguatge escrit, sempre que tinga un bon control motor. No obstant açò, la majoria tenen dificultats per a usar sistemes de Comunicació Augmentatius i Alternatius (AAC). De fet, les persones amb DCP utilitzen generalment taulers de conceptes per a indicar la idea que volen transmetre. No obstant açò, la majoria les solucions de comunicació disponibles en l'actualitat estan basades en programari propietari que fan difícil la personalització del tauler de conceptes i el tipus de sistema de control. És ací on sorgeix aquesta tesi, amb l'objectiu de crear una interfície amb aqueixes característiques, capaç d'adaptar-se a les necessitats i limitacions de l'usuari. D'aquesta forma, aquesta tesi proposa un sistema de comunicació augmentatiu i alternatiu per a persones amb DCP basat en control fisiològic. A més, es proposa un sistema innovador de control directe sobre taulers de conceptes basat en EMG. Aquest sistema de control es basa en un model físic que reprodueix la resposta mecànica muscular (basat en paràmetres com a Rigidesa, Inèrcia i Viscositat), permetent la selecció d'elements gràcies a xicotets polsos de senyal EMG amb sensors sobre un múscul amb control motor. Els seus principals avantatges són la possibilitat de corregir errors durant la selecció associat als impulsos musculars no controlats, evitar l'esforç muscular mantingut per a aconseguir un nivell i reduir la fatiga.Díaz Pineda, JA. (2017). Advanced Augmentative and Alternative Communication System Based in Physiological Control [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/90418TESI

Diseño de un sistema de detección y clasificación de cambios emocionales basado en el análisis de señales fisiológicas no intrusivas

176 p.Aunque la autonomía y la independencia de las personas son valores inherentes en los sereshumanos, existen colectivos que no pueden disfrutar plenamente de ellos, como es el caso de:personas de la tercera edad, enfermos crónicos, personas con determinada discapacidadintelectual, etc. Promover la independencia de estas personas, tanto en ambientes laboralescomo sociales, es uno de los principales aspectos para mejorar su calidad de vida y la de losfamiliares y tutores que los asisten. El estudio de las emociones humanas y sus respuestas antedeterminados eventos, es un paso importante para avanzar en este camino.Dentro de los trabajos cuyo enfoque es el de proporcionar apoyo a las personas conautonomía reducida, el presentado en esta investigación tiene como fin el desarrollo de unalgoritmo que permita detectar cambios emocionales a partir de la lectura de variablesfisiológicas recogidas de forma no invasiva, como son la variabilidad del ritmo cardíaco (HRV) yla respuesta galvánica de la piel (GSR).Esta propuesta de tesis presenta un sistema de clasificación emocional en tiempo real basadoen máquinas de estados finitos, a partir del análisis de la HRV y GSR. El algoritmo desarrolladodetecta la activación del sistema nervioso simpático relacionada con estados de alerta y estrés,y la inhibición del mismo asociada a emociones como el bienestar y la tristeza sin lloro.Con el objeto de mejorar las propuestas ya existentes, el sistema tiene la capacidad de graduarla activación e inhibición simpática en tres niveles: baja, media o alta, haciendosimultáneamente una clasificación del tipo de activación y etiquetándola como estréscontinuado o alerta momentánea.Se realizaron cuatro experimentos con el objeto de disponer de una base de datos de señalesfisiológicas asociadas a cambios emocionales. Para ello se diseñaron varios experimentos queen condiciones de laboratorio, permitan elicitar de la forma más real posible emocionesbásicas (enfado, bienestar, diversión, sorpresa, asco, miedo y tristeza) y estados de estrés.Para medir el ratio de acierto del algoritmo en la identificación de las emociones llevadas aestudio, se ha utilizado el parámetro F1¿score. Los resultados obtenidos tras aplicar el sistemaa base de datos, muestran una precisión de 0.98 para detectar estados de activación alta, 0.97para media y 0.94 para baja. La precisión obtenida en la detección de estados de inhibición esde 1.00 en la emoción sorpresa y 0.987 en bienestar. Una vez analizados los resultados, sepuede afirmar que se ha diseñar una herramienta con elevados ratios de acierto en ladetección y clasificación de estados emocionales, basada en el estudio de la activación einhibición del sistema nervioso simpático

Sensitivity analysis in a scoping review on police accountability : assessing the feasibility of reporting criteria in mixed studies reviews

In this paper, we report on the findings of a sensitivity analysis that was carried out within a previously conducted scoping review, hoping to contribute to the ongoing debate about how to assess the quality of research in mixed methods reviews. Previous sensitivity analyses mainly concluded that the exclusion of inadequately reported or lower quality studies did not have a significant effect on the results of the synthesis. In this study, we conducted a sensitivity analysis on the basis of reporting criteria with the aims of analysing its impact on the synthesis results and assessing its feasibility. Contrary to some previous studies, our analysis showed that the exclusion of inadequately reported studies had an impact on the results of the thematic synthesis. Initially, we also sought to propose a refinement of reporting criteria based on the literature and our own experiences. In this way, we aimed to facilitate the assessment of reporting criteria and enhance its consistency. However, based on the results of our sensitivity analysis, we opted not to make such a refinement since many publications included in this analysis did not sufficiently report on the methodology. As such, a refinement would not be useful considering that researchers would be unable to assess these (sub-)criteria

Computational approaches to Explainable Artificial Intelligence: Advances in theory, applications and trends

Deep Learning (DL), a groundbreaking branch of Machine Learning (ML), has emerged as a driving force in both theoretical and applied Artificial Intelligence (AI). DL algorithms, rooted in complex and non-linear artificial neural systems, excel at extracting high-level features from data. DL has demonstrated human-level performance in real-world tasks, including clinical diagnostics, and has unlocked solutions to previously intractable problems in virtual agent design, robotics, genomics, neuroimaging, computer vision, and industrial automation. In this paper, the most relevant advances from the last few years in Artificial Intelligence (AI) and several applications to neuroscience, neuroimaging, computer vision, and robotics are presented, reviewed and discussed. In this way, we summarize the state-of-the-art in AI methods, models and applications within a collection of works presented at the 9 International Conference on the Interplay between Natural and Artificial Computation (IWINAC). The works presented in this paper are excellent examples of new scientific discoveries made in laboratories that have successfully transitioned to real-life applications

Computational Approaches to Explainable Artificial Intelligence:Advances in Theory, Applications and Trends

Deep Learning (DL), a groundbreaking branch of Machine Learning (ML), has emerged as a driving force in both theoretical and applied Artificial Intelligence (AI). DL algorithms, rooted in complex and non-linear artificial neural systems, excel at extracting high-level features from data. DL has demonstrated human-level performance in real-world tasks, including clinical diagnostics, and has unlocked solutions to previously intractable problems in virtual agent design, robotics, genomics, neuroimaging, computer vision, and industrial automation. In this paper, the most relevant advances from the last few years in Artificial Intelligence (AI) and several applications to neuroscience, neuroimaging, computer vision, and robotics are presented, reviewed and discussed. In this way, we summarize the state-of-the-art in AI methods, models and applications within a collection of works presented at the 9 International Conference on the Interplay between Natural and Artificial Computation (IWINAC). The works presented in this paper are excellent examples of new scientific discoveries made in laboratories that have successfully transitioned to real-life applications

4th International Symposium on Ambient Intelligence (ISAmI 2013)

Ambient Intelligence (AmI) is a recent paradigm emerging from Artificial Intelligence (AI), where computers are used as proactive tools assisting people with their day-to-day activities, making everyone’s life more comfortable. Another main concern of AmI originates from the human computer interaction domain and focuses on offering ways to interact with systems in a more natural way by means user friendly interfaces. This field is evolving quickly as can be witnessed by the emerging natural language and gesture based types of interaction. The inclusion of computational power and communication technologies in everyday objects is growing and their embedding into our environments should be as invisible as possible. In order for AmI to be successful, human interaction with computing power and embedded systems in the surroundings should be smooth and happen without people actually noticing it. The only awareness people should have arises from AmI: more safety, comfort and wellbeing, emerging in a natural and inherent way. ISAmI is the International Symposium on Ambient Intelligence and aiming to bring together researchers from various disciplines that constitute the scientific field of Ambient Intelligence to present and discuss the latest results, new ideas, projects and lessons learned, namely in terms of software and applications, and aims to bring together researchers from various disciplines that are interested in all aspects of this area

‘The shops were only made for people who could walk’: impairment, barriers and autonomy in the mobility of adults with Cerebral Palsy in urban England

Based on research carried out with a group of adults with Cerebral Palsy in Birmingham, UK, we consider the complex inter-relationship between the accessibility of the urban environment for those with impaired gross motor skills, and the ability of these people to lead full and independent lives. Drawing on a framework that considers mobility as movement, meaning-making and political, we demonstrate the reality of differentiated mobility. For those with bodies that function outside the presumed operating parameters of the model subjects of urban design, mobility may be possible, but is often uncomfortable and even dangerous, with significant associated effects for impaired people’s autonomy. Our study details social and structural, or design, barriers to people’s mobility, demonstrating the inter-connection between individuals’ behaviour and urban design in a manner that questions a clear distinction between the two. We draw upon the notions of emotional work and a commoning approach to mobility in suggesting that further investment in urban accessibility is squarely an issue of social justice