Emotional expressions reconsidered: challenges to inferring emotion from human facial movements

It is commonly assumed that a person’s emotional state can be readily inferred from his or her facial movements, typically called emotional expressions or facial expressions. This assumption influences legal judgments, policy decisions, national security protocols, and educational practices; guides the diagnosis and treatment of psychiatric illness, as well as the development of commercial applications; and pervades everyday social interactions as well as research in other scientific fields such as artificial intelligence, neuroscience, and computer vision. In this article, we survey examples of this widespread assumption, which we refer to as the common view, and we then examine the scientific evidence that tests this view, focusing on the six most popular emotion categories used by consumers of emotion research: anger, disgust, fear, happiness, sadness, and surprise. The available scientific evidence suggests that people do sometimes smile when happy, frown when sad, scowl when angry, and so on, as proposed by the common view, more than what would be expected by chance. Yet how people communicate anger, disgust, fear, happiness, sadness, and surprise varies substantially across cultures, situations, and even across people within a single situation. Furthermore, similar configurations of facial movements variably express instances of more than one emotion category. In fact, a given configuration of facial movements, such as a scowl, often communicates something other than an emotional state. Scientists agree that facial movements convey a range of information and are important for social communication, emotional or otherwise. But our review suggests an urgent need for research that examines how people actually move their faces to express emotions and other social information in the variety of contexts that make up everyday life, as well as careful study of the mechanisms by which people perceive instances of emotion in one another. We make specific research recommendations that will yield a more valid picture of how people move their faces to express emotions and how they infer emotional meaning from facial movements in situations of everyday life. This research is crucial to provide consumers of emotion research with the translational information they require

The role of learning theory in multimodal learning analytics

This study presents the outcomes of a semi-systematic literature review on the role of learning theory in multimodal learning analytics (MMLA) research. Based on previous systematic literature reviews in MMLA and an additional new search, 35MMLA works were identified that use theory. The results show that MMLA studies do not always discuss their findings within an established theoretical framework. Most of the theory-driven MMLA studies are positioned in the cognitive and affective domains, and the three most frequently used theories are embodied cognition, cognitive load theory and control–value theory of achievement emotions. Often, the theories are only used to inform the study design, but there is a relationship between the most frequently used theories and the data modalities used to operationalize those theories. Although studies such as these are rare, the findings indicate that MMLA affordances can, indeed, lead to theoretical contributions to learning sciences. In this work, we discuss methods of accelerating theory-driven MMLA research and how this acceleration can extend or even create new theoretical knowledge

The Effects of Postpartum Depression on Children\u27s Social Development

The increased incidence of postpartum depression has had significant effects on children’s social development. The purpose of this systematic review is to bring attention to the growing problem in such a vulnerable population. In addition, it was designed to shed light on the lack of research in this area of healthcare. The methods used to conduct the study include various peer reviewed, scholarly and evidenced based articles from databases such as Academic Search Complete, PsycNet, and Pubmed. Each article has been critically evaluated based on the following guidelines: a population group of children under the age of four, specifically maternal postpartum depression rather than paternal, and studies focused on childhood social development. The general consensus of the twenty articles conclude that maternal postpartum depression disrupts the social development of children. Specifically, decreased levels of attachment have been a common trend along with a developmental delay of communication. Based on the evidence collected during the systematic review future evidence-based practice should involve more rigorous screening of the mother child dyad in relation to promotion of mental health. How are children internationally, from birth to four years old, impacted by postpartum depression in relation to social development? Keywords: postpartum depression, development, social development, cognitive development, pediatrics, mental health, infan

Assessment of Emotion Understanding in Preschoolers: Multiple-Choice vs. Open-Ended Methods

This study examines a new method of assessing emotion understanding in preschoolers. Prior research has established that preschool is a critical time for emotion understanding development and that emotion understanding abilities are related to such important outcomes as social competence. Traditionally, measures of emotion understanding present multiple-choice questions that require children to select one emotion that is most likely to be elicited in various situations. However, this study proposes an alternative method in which children are asked to explain their answers on a subset of items. Their open-ended responses are then coded for quality of reasoning. Results establish preliminary evidence of the reliability and validity of the new assessment method. Notably, multiple regression analyses indicate that the coded emotion reasoning scores are a better predictor of social competence than scores obtained using traditional multiple-choice procedures. Limitations of the study and implications for future research are also discussed

인공지능 기반 교육 플랫폼 사용에 대한 중국 교사의 인식

학위논문 (석사) -- 서울대학교 대학원 : 사범대학 교육학과, 2021. 2. 조영환.최근 교육 분야에서 인공지능(AI)의 도입이 큰 관심을 끌고 있다. 특히 AI 기술과 학습 분석이 결합한 인공지능 기반 교육 플랫폼은 지금껏 실현되기 어려웠던 맞춤형 학습(personalized learning)과 적응적 학습(adaptive learning)에 도움이 될 수 있도록 발전하고 있다. 인공지능 기반 교육 플랫폼(AI-based education platform)은 학습자의 행동 추적 등을 통해 이들의 특성을 분석하고 진단을 제공한 뒤 분석 결과를 토대로 학습자에게 인지 수준에 맞는 맞춤형 학습자원과 피드백을 제공한다. 인공지능 기반 교육 플랫폼은 교사와 학생에게 실시간 학습 데이터와 분석 결과, 그리고 피드백을 제공할 수 있어 인공지능 기반 교육 플랫폼이 맞춤형 학습에 긍정적인 의미가 있다는 선행 연구도 있었다. 그럼에도 불구하고, 기존 연구는 모델 개발의 차원에서나 엄밀한 실험실 환경에서 인공지능 기반 교육 플랫폼의 효과를 연구해왔으며, 인공지능 기반 교육 플랫폼에 대한 교사의 인식과 관련된 연구는 드물었다. 교사는 인공지능 교육 기술의 사용자이기 때문에 인공지능 교육 기술의 교육 도입에 있어 교사들의 인식과 의견은 중요하다. 본 연구는 인공지능 기반 교육 플랫폼을 활용하는 것에 대한 교사들의 인식을 탐구하였다. 아래 연구 문제를 다루기 위해 질적 연구를 시행하였다. 첫째, 중국 교사들은 인공지능 기반 교육 플랫폼이 중학교 교육에 활용 있어 어떠한 장점이 있다고 인식하는가? 둘째, 중국 교사들은 인공지능 기반 교육 플랫폼과 중학교 교수 활동 요소 간 어떠한 모순이 있다고 인식하는가? 셋째, 중국 교사들은 인공지능 기반 교육 플랫폼을 중학교 교육에 도입할 때 무엇이 필요하다고 인식하는가? 본 연구는 중국 교사들을 연구대상으로 온라인 심층 면담을 하였다. 문헌 리뷰를 통해 면담 질문지를 설계하되 눈덩이표집법 (snowball sampling)을 통해 중국 중학교 교사 14명을 연구참여자로 선정하였다. 선정된 교사들은 모두 인공지능 기반 교육 플랫폼 사용 경험이 있으며 각 교사를 대상으로 약 1시간 정도 면담을 진행하고 녹음하였다. 면담이 끝난 후 녹음 내용을 전사하였으며, 주제분석을 사용하여 면담 내용을 초기 코드 생성하고 면담 자료 속에서 주제를 도출하였다. 특히 연구 문제 2번의 경우, 인공지능 기반 교육 플랫폼 활용과 교수 학습활동 내 여러 요소 간의 모순을 분석하기 위해 활동이론을 연구의 틀로 이용하였다. 최종적으로 연구문제 1에 대한 주제 4개, 연구문제 2에 대한 주제 6개, 연구문제 3에 대한 주제 4개를 도출하였다. 연구 결과로 교사들은 인공지능 기반 교육 플랫폼의 장점에 대해 즉각적인 피드백 제공, 교수학습 지원, 교사의 업무량 감소 등으로 인식하였고, 인공지능 기반 교육 플랫폼이 다양한 교수학습 자원을 통합할 수 있다고 인식하였다. 아울러 교사들은 인공지능 기반 교육 플랫폼의 사용에 있어 기존의 교수학습 활동과 상충된 부분이 있다는 점을 인식하였다. 교사들은 기존 인공지능 기반 교육 플랫폼의 추천 모델이 차별화된 학생들에게 잘 적용되지 못한다는 것을 인식하였다. 그리고 기존 인공지능 기반 교육 플랫폼이 다양한 학습 자원을 잘 분류되지 못하기 때문에 교사들이 사용하기 불편하다. 인공지능 기반 교육 플랫폼을 이용할 때 교사의 지적재산권을 보호하기 위한 명확한 규제가 부족하다고 인식하였다. 이와 함께 학부모들은 인공지능 기반 교육 플랫폼을 사용함으로써 발생할 수 있는 학습자의 인터넷 남용과 시력 저하 문제를 우려하였다. 또 중국의 사회문화적 배경과 교육 특성으로 인해 인공지능 기반 교육 플랫폼을 활용하는 데 학생들의 글씨 쓰기 능력에 영향을 미칠 수 있으며, 학교 내 전자기기 사용 제한도 데이터 수집의 지속성과 효율성에 영향을 미칠 수 있다고 인식하였다. 교사들은 위의 문제들이 인공지능 교육 플랫폼 사용에 대한 규칙 마련과 인공지능 기술을 개선함으로써 완화될 수 있다고 인식하였다. 또한 교사의 실제 요구에 맞게 개발될 수 있도록 인공지능 기반 교육 플랫폼 개발 과정에 교육 전문가와 교사가 참여할 필요가 있다. 본 연구는 중국 교사들이 인공지능 기반 교육 플랫폼에 대한 인식을 탐색하였으며, 인공지능 기반 교육 플랫폼이 교수학습에서의 장점과 문제점을 밝혔다. 아울러 본 연구는 인공지능 기반 교육 플랫폼이 교육 분야에 대규모로 도입될 수 있도록 규칙, 인공지능 기술, 그리고 교육 공학의 차원에서 사용 규범과 기술 개선을 제안하였다. 본 연구를 통해 탐색한 내용이 향후 교육 분야의 인공지능 기반 교육 플랫폼 도입에 활용된다면 인공지능 교육 기술에 관한 연구의 발전에도 기여할 수 있을 것으로 기대된다.In recent years, the introduction of artificial intelligence (AI) in education has attracted widespread attention. In particular, the AI-based education platform based on the combination of AI technology and learning analysis brings new light to the long-standing difficulties in personalized learning and adaptive learning. The AI-based education platform analyzes learners' characteristics by collecting their data and tracking their learning behavior. It then generates cognitive diagnosis for learners and provides them with personalized learning resources and adaptive feedback that match their cognitive level based on systematic analysis. With the help of the AI-based education platform, teachers and students can get real-time educational data and analysis result，as well as the feedback and treatment corresponding to the results. Previous studies have already demonstrated and proved its positive significance to personalized learning. However, these studies mostly start from a model development perspective or in a rigorous laboratory environment. There has been little research on teachers' perceptions of AI-based education platform. As a direct user of AI educational technologies, teachers' perceptions and suggestions are vital for introducing AIEd in education. In this study, the researcher explored teachers' perceptions of using AI-based education platform in teaching. The study conducted qualitative research to address the following research questions: 1) How do Chinese teachers perceive the advantages of AI-based education platforms for teaching and learning in secondary school? 2) How do Chinese teachers perceive the contradictions between AI-based education platforms and the secondary school system? 3）How do Chinese teachers suggest applying AI-based education platforms in secondary school? And it referred to the in-depth online interview with Chinese teachers who had experience with AI-based education platform. Interview questions were constructed through the literature review, and 14 secondary school teachers were selected by the snowball sampling method. The interviews lasted for an average of one hour per teacher and were transcribed from the audio recordings to text documents when finished. Afterward, the data were analyzed using thematic analysis, including generating initial codes, searching and reviewing the categories, and deriving the themes finally. Notably, for research question two, the researcher used the activity theory framework to analyze the contradictions among the use of the AI-based education platform and the various elements of the teaching and learning activities. Finally, four themes for research question 1, six themes for research question 2, and four themes for research question 3 were derived. As for the advantages, teachers believe that AI-based education platforms can provide instant feedback, targeted and systematic teaching support, and reduce teachers' workload. At the same time, AI-based education platforms can also integrate teaching resources in different areas. Teachers also recognized that the AI-based education platforms might trigger contradictions in existing teaching activities. They are aware of the situation that the recommended model of the AI-based education platform is not suitable for all levels of students; that a large number of learning resources are not classified properly enough to meet the needs of teachers, and that there lack clear rules and regulations to protect teachers' intellectual property rights when using the platform. Besides, parents are also concerned about the potential risk of internet addiction and vision problems using AI-based education platforms. Moreover, the use of the AI-based education platform may also affect students' ability to write Chinese characters due to the socio-historical background and educational characteristics in China. Furthermore, the restricted use of electronic devices on campus may also impact the consistent and effective education data collection. Teachers believe that these problems can be solved by improving rules and AI technology. Moreover, to make the platform more in line with the actual teaching requirements, teachers and education experts can also be involved in the development process of AI-based education platform. This study explored how Chinese teachers perceive the AI-based education platform and found that the AI-based education platform was conducive to personalized teaching and learning. At the same time, this study put forward some suggestions from the perspective of rules, AI technology, and educational technology, hoping to provide a good value for the future large-scale introduction of AI-based education platforms in education.CHAPTER 1. INTRODUCTION 1 1.1. Problem Statement 1 1.2. Purpose of Research 7 1.3. Definition of Terms 8 CHAPTER 2. LITERATURE REVIEW 10 2.1. AI in Education 10 2.1.1 AI for Learning and Teaching 10 2.1.2 AI-based Education Platform 14 2.1.3 Teachers' Perception on AI-based Education Platform 18 2.2. Activity Theory 20 CHAPTER 3. RESEARCH METHOD 23 3.1. Research Design 23 3.2. Participants 25 3.3. Instrumentation 26 3.3.1 Potential Value of AI System in Education 26 3.4. Data Collection 33 3.5. Data Analysis 34 CHAPTER 4. FINDINGS 36 4.1. Advantages of Using AI-based Education Platform 36 4.1.1 Instant Feedback 37 4.1.2 Targeted and Systematic Teaching Support 42 4.1.3 Educational Resources Sharing 46 4.1.4 Reducing Workload 49 4.2. Tensions of Using AI-based Education Platform 51 4.2.1 Inadequately Meet the Needs of Teachers 52 4.2.2 Failure to Satisfy Low and High Achievers 54 4.2.3 Intellectual Property Violation 56 4.2.4 Guardian's Concern 57 4.2.5 School Rules about the Use of Electronic Devices 58 4.2.6 Implication for Chinese Character Education 59 4.3. Suggestion of Using AI-based Education Platform 61 4.3.1 Improving Rules of Using the AI-based Education Platform 61 4.3.2 Improving Rules of Protecting Teachers Right 62 4.3.3 Improving AI Technology 64 4.3.4 Participatory Design 66 CHAPTER 5. DISCUSSION AND CONCLUSION 68 5.1. Discussion 68 5.2. Conclusion 72 REFERENCE 75 APPENDIX 1 98 APPENDIX 2 100 국문초록 112Maste

Infant attentional markers studied with eye-tracking as possible predictors of toddler age adaptive behavior

Objectives: Early recognition of developmental challenges in infancy is central for being able to start interventions earlier with possible better outcomes. The aim of this study was to investigate the potential of using eye-tracking at seven months to predict adaptive behavior outcomes at two years. Adaptive behavior implies the child's adaptation to their specific living environment and it includes aspects of conceptual, social and practical adaptation. Here, I studied several eye-tracking based markers of visual attention in infancy for their possible association with adaptive behavior at two years, such as saccadic reaction time (SRT) and measures of attentional preference for faces and emotional facial expressions. SRT has been shown to be a general attentional marker in infancy associated with many developmental and clinical aspects while attentional preference for faces and emotional facial expressions are pronounced at seven months and have been hypothesized to play a role in socio-emotional development. Infants were also studied with respect to attentional behavioral tendencies based on their SRT measures. Methods: This study employed material from the Toibilas-study at BABA center at Helsinki University Hospital. The study consisted of 45 children of either typical development (n=26) or children of high developmental risk (n=19) due to their prior treatment at the neonatal intensive care unit. The high-risk group was assumed to increase variability of the sample. Infants participated in eye-tracking measurements at seven months old using two different eye-tracking paradigms; a SRT task and a task measuring attentional bias for facial vs. non-facial as well as emotional facial expressions vs. neutral facial expressions. The same children were assessed for adaptive behavior at two years of age by a parent filled-questionnaire. Associations between attentional behavioral markers at seven months and adaptive behavior at two years were analyzed using linear models. The models also accounted for behavioral attentional tendency as identified from a set of SRT measures. Results and conclusions: A bigger attentional preference for faces compared to non-faces was associated with better overall adaptive behavior score as well as adaptive behavior composites for conceptual and practical adaptive behavior. This is in line with the hypothesis that attentional bias for faces is important for socio-emotional development. A significant association between identified behavioral attentional tendencies and the conceptual composite score of adaptive behavior was found. This finding suggests that recognizing attentional tendency may be useful for predicting later developmental trajectories. The present work provides preliminary evidence for the proof of principle that eye-tracking based metrics may provide clinically relevant predictions in infants, however novel prospective datasets are needed for clinical validation.Målsättning: Identifiering av utvecklingsmässiga utmaningar redan i spädbarnsåldern är viktigt för att ha möjlighet att inleda tidiga interventioner och därmed potentiellt öka interventionernas effektivitet. Målsättningen för den här studien var att utreda användbarheten av ögonrörelsemätning (eye-tracking) i sju månaders ålder för att förutspå adaptivt beteende i två års ålder. Adaptivt beteende beskriver barnets anpassning till hans eller hennes specifika livsmiljö och innehåller begreppsmässig, social och praktisk anpassning. I den här studien har jag använt flera markörer för visuell uppmärksamhet mätta i spädbarns åldern m.hj.a ögonrörelsemätning; sackadiska ögonrörelsers reaktionshastighet (SRT) samt preferens att rikta uppmärksamhet mot ansikten och emotionella ansiktsuttryck. Jag har undersökt de olika markörernas association till adaptivt beteende vid två års ålder. SRT är ett allmänt mått för visuell uppmärksamhet i spädbarnsåldern och har förknippats med flera utvecklingsmässiga och kliniska aspekter medan uppmärksamhetsmässig preferens för ansikten och emotionella ansiktsuttryck är markant vid sju månaders ålder och har föreslagits ha en roll i socio-emotionell utveckling. Möjlig association mellan spädbarnens tendenser för uppmärksamhetsbeteende, identifierat m. hj. a. flera olika SRT-mått, och adaptivt beteende undersöktes också. Metoder: Den här studien använde sig av material från Toibilas-studien vid BABA-center, Helsingfors Universitets Sjukhus. Studien innefattade 45 barn, både barn utan identifierade utvecklingsmässiga risker (n=26) och barn tillhörande en hög-risk grupp (n=19) p.g.a. deras tidigare vårdperiod vid neonatal intensivvård avdelningen. Vid sju månaders ålder deltog spädbarnen i en ögonrörelsemätning med två olika uppgifter; en SRT-uppgift och en uppgift som mätte skillnaden i uppmärksamhet gentemot ansikten jämfört med annat visuellt stimulus samt emotionella ansiktsuttryck jämfört med neutrala ansiktsuttryck. Samma barns adaptiva beteende utvärderades vid två års ålder med ett frågeformulär ifyllt av en förälder. Associationen mellan markörer för visuell uppmärksamhet, barnens tendenser för uppmärksamhetsbeteende i spädbarnsåldern och adaptivt beteende i två års ålder undersöktes med lineära modeller. Resultat och slutsatser: Större preferens för ansikten jämfört med annat visuellt stimulus vid sju månaders ålder var förknippat med bättre adaptivt beteende vid två års ålder. Upptäckten stöder hypotesen att preferensen för uppmärksamhet riktad mot ansikten är viktig för den socio-emotionella utvecklingen. En signifikant association mellan spädbarnets tendenser för uppmärksamhetsbeteende och begreppsmässigt adaptivt beteende kunde konstateras. Detta fynd tyder på att identifikation av spädbarnets tendenser för uppmärksamhetsbeteende kunde vara nyttigt för att förutspå senare utveckling. Denna studie ger preliminära bevis för principen att ögonrörelsemätning i spädbarnsåldern kunde ge kliniskt relevant infromation som förutspår utveckling, men upprepning av resultaten och framtida forskning för klinisk verifiering är av yttersta vikt

Multisensory Perception and Learning: Linking Pedagogy, Psychophysics, and Human–Computer Interaction

In this review, we discuss how specific sensory channels can mediate the learning of properties of the environment. In recent years, schools have increasingly been using multisensory technology for teaching. However, it still needs to be sufficiently grounded in neuroscientific and pedagogical evidence. Researchers have recently renewed understanding around the role of communication between sensory modalities during development. In the current review, we outline four principles that will aid technological development based on theoretical models of multisensory development and embodiment to foster in-depth, perceptual, and conceptual learning of mathematics. We also discuss how a multidisciplinary approach offers a unique contribution to development of new practical solutions for learning in school. Scientists, engineers, and pedagogical experts offer their interdisciplinary points of view on this topic. At the end of the review, we present our results, showing that one can use multiple sensory inputs and sensorimotor associations in multisensory technology to improve the discrimination of angles, but also possibly for educational purposes. Finally, we present an application, the ‘RobotAngle’ developed for primary (i.e., elementary) school children, which uses sounds and body movements to learn about angles

Collecting Data from Children Ages 9-13

Provides a summary of literature on common methods used to collect data, such as diaries, interviews, observational methods, and surveys. Analyzes age group-specific considerations, advantages, and drawbacks, with tips for improving data quality

Capturing Primary School Students’ Emotional Responses with a Sensor Wristband

The emotions experienced by primary school students have both positive and negative effects on learning processes. Thus, to better understand learning processes, research should consider emotions during class. Standard survey-based methods, such as self-reports, are limited in terms of capturing the detailed trajectories of primary school  children’s emotions, as their abilities of self-reporting are developing and still limited. Emotions can also be tracked by capturing emotional responses as they occur e.g. from physiological reaction measured with sensor wristbands. This technology generates an emotional responsestypology based on continuously captured physiological data, such as skin conductivity and skin temperature. However, such measurement methods need to be validated before being used. The present study thus attempted to validate this instrument with primary school students. We used the BM Sensor Wristband technology, as its emotional response typology is based on the categorical emotion and homeostasis approach. In our research, we focus on the emotional responses that can be distinguished by the BM Typology and that can influence learning processes. These emotional responses are: “joy”, “curiosity”, “attention”, “fear”, “anger” and “passivity”. Therefore, we induced emotional responses in primary school children through specifically developed audio-visual stimuli. Using logistic mixed effects modelling, we investigated the occurrence of opposing reactions. We observed that primary school children’s reactions to audio-visual stimuli could be differentiated. We conclude that primary school children’s emotional responses, such as “joy”, “curiosity”, “attention”, “fear”, “anger” and “passivity”, can be accurately measured by evaluating physiological data