Multimodal Learning For Classroom Activity Detection

Classroom activity detection (CAD) focuses on accurately classifying whether the teacher or student is speaking and recording both the length of individual utterances during a class. A CAD solution helps teachers get instant feedback on their pedagogical instructions. This greatly improves educators' teaching skills and hence leads to students' achievement. However, CAD is very challenging because (1) the CAD model needs to be generalized well enough for different teachers and students; (2) data from both vocal and language modalities has to be wisely fused so that they can be complementary; and (3) the solution shouldn't heavily rely on additional recording device. In this paper, we address the above challenges by using a novel attention based neural framework. Our framework not only extracts both speech and language information, but utilizes attention mechanism to capture long-term semantic dependence. Our framework is device-free and is able to take any classroom recording as input. The proposed CAD learning framework is evaluated in two real-world education applications. The experimental results demonstrate the benefits of our approach on learning attention based neural network from classroom data with different modalities, and show our approach is able to outperform state-of-the-art baselines in terms of various evaluation metrics.Comment: The 45th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2020

Learning Multi-level Dependencies for Robust Word Recognition

Robust language processing systems are becoming increasingly important given the recent awareness of dangerous situations where brittle machine learning models can be easily broken with the presence of noises. In this paper, we introduce a robust word recognition framework that captures multi-level sequential dependencies in noised sentences. The proposed framework employs a sequence-to-sequence model over characters of each word, whose output is given to a word-level bi-directional recurrent neural network. We conduct extensive experiments to verify the effectiveness of the framework. The results show that the proposed framework outperforms state-of-the-art methods by a large margin and they also suggest that character-level dependencies can play an important role in word recognition

Localization of CO2 leakage from transportation pipelines through low frequency acoustic emission detection

Carbon Capture and Storage is a technology to reduce greenhouse gas emissions. CO2 leak from high pressure CO2 transportation pipelines can pose a significant threat to the safety and health of the people living in the vicinity of the pipelines. This paper presents a technique for the efficient localization of CO2 leakage in the transportation pipelines using acoustic emission method with low frequency and narrow band sensors. Experimental tests were carried out on a lab scale test rig releasing CO2 from a stainless steel pipe. Further, the characteristics of the acoustic emission signals are analyzed in both the time and the frequency domains. The impact of using the transverse wave speed and the longitudinal wave speed on the accuracy of the leak localization is investigated. Since the acoustic signals are expected to be attenuated and dispersed when propagating along the pipe, empirical mode decomposition, signal reconstruction and a data fusion method are employed in order to extract high quality data for accurate localization of the leak source. It is demonstrated that a localization error of approximately 5% is achievable with the proposed detecting system