Multimodal Learning For Classroom Activity Detection

Classroom activity detection (CAD) focuses on accurately classifying whether the teacher or student is speaking and recording both the length of individual utterances during a class. A CAD solution helps teachers get instant feedback on their pedagogical instructions. This greatly improves educators' teaching skills and hence leads to students' achievement. However, CAD is very challenging because (1) the CAD model needs to be generalized well enough for different teachers and students; (2) data from both vocal and language modalities has to be wisely fused so that they can be complementary; and (3) the solution shouldn't heavily rely on additional recording device. In this paper, we address the above challenges by using a novel attention based neural framework. Our framework not only extracts both speech and language information, but utilizes attention mechanism to capture long-term semantic dependence. Our framework is device-free and is able to take any classroom recording as input. The proposed CAD learning framework is evaluated in two real-world education applications. The experimental results demonstrate the benefits of our approach on learning attention based neural network from classroom data with different modalities, and show our approach is able to outperform state-of-the-art baselines in terms of various evaluation metrics.Comment: The 45th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2020

Learning Multi-level Dependencies for Robust Word Recognition

Robust language processing systems are becoming increasingly important given the recent awareness of dangerous situations where brittle machine learning models can be easily broken with the presence of noises. In this paper, we introduce a robust word recognition framework that captures multi-level sequential dependencies in noised sentences. The proposed framework employs a sequence-to-sequence model over characters of each word, whose output is given to a word-level bi-directional recurrent neural network. We conduct extensive experiments to verify the effectiveness of the framework. The results show that the proposed framework outperforms state-of-the-art methods by a large margin and they also suggest that character-level dependencies can play an important role in word recognition

Efficacy and safety of switching to flumatinib in patients with chronic myeloid leukemia who have not achieved optimal response or are intolerant to TKI treatment

Objective To observe the efficacy and safety of flumatinib conversion in chronic myelogenous leukemia-chronicphase (CML-CP) patients with suboptimal TKI response or intolerance. Methods Patients who did not have the best response or intolerance to first-line imatinib, dasatinib, and nilotinib and switched to flumatinib (600 mg/d) from February 2020 to August 2022 were collected from 5 hospitals from Chongqing and affiliated hospitals of North Sichuan Medical College.The efficacy and safety of flumatinib were observed.The optimal response rate, major molecular response (MMR), cumulative complete cytogenetic response (CCyR) rate, cumulative MMR rate, cumulative deep molecular response (DMR), progression-free survival (PFS), event-free survival (EFS) and adverse reactions in 3, 6 and 12 months after treatment were observed and analyzed. Results A total of 100 patients with CML-CP were enrolled, with a median follow-up of 18(3~36) months.The optimal response rate was 92.6%(88/95), 94.4%(85/90) and 92.9%(79/85) respectively, at 3, 6 and 12 months after treatment.Till August 20, 2023, the cumulative CCyR and MMR rate was 98.0%(98/100) and 81.9%(77/94), respectively, the median time to reach CCyR and MMR was 3 months, and cumulative DMR rate was 51.0%(51/100).PFS rate was 100.0%(100/100) and 1-year EFS rate was 85.6%(75/90).The most common non-hematologic adverse reactions of flumatinib were diarrhea and abdominal pain (7.0%), followed by renal dysfunction (6.0%) and musculoskeletal pain (2.0%).The main hematologic adverse reactions were thrombocytopenia (12.0%), anemia (6.0%) and leukopenia (2.0%). Conclusion Flumatinib has better MMR and DMR and is well tolerated in CML-CP patients with TKI resistance or intolerance

What Can Spontaneous Facial Expression Tell Us?

Facial expression plays a significant role in human communication. It is considered the single most important cue in the psychology of emotion. Facial expression is taken as a universally understood signal, which triggers a discrete categorical basic emotion, including joy, sadness, fear, surprise, anger, and disgust. Thus, automatic analysis of emotion from images of human facial expression has been an interesting and challenging problem for the past 30 years. Aiming towards the applications of human behavior analysis, human-human interaction and human-computer interaction, this topic has recently drawn even more attention.Automatic analysis of facial expression in a realistic scenario is a much more difficult problem due to that the 2-D imagery of human facial expression consists of rigid head motion and non-rigid muscle motion. We are tasked to solve this "coupled-motion" problem and analyze facial expression in a meaningful manner. We first proposed an image-based representation, Emotion Avatar Image, to help person-independent expression recognition. Second, an real-time registration technique is designed to improve frame-based streaming action unit (AU) recognition. The proposed accurate expression recognition techniques are then applied to the field of advertising, where audiences' commercial watching behavior is thoroughly analyzed