Improvement of Student Attention Monitoring Supported by Precision Sensing in Learning Management Systems

A Learning Management Systems (LMS) can benefit from the inclusion Computer-Mediated-Communications (CMC) software for delivering materials. Incorporating CMC tools in virtual classrooms or implementing educational blogs, can be very effective in e-learning platforms. In such student-centered interaction scenarios, it is important to monitor and manage student attention in a precise way to enhance student performance. Sensing with precision through 6G/7G technology allows to include electronic and software devices to produce such monitoring. This chapter contextualizes and describes an abstraction application scenario of sensing and monitoring student attention with high precision in Learning Management System with new communication systems. In that context, technology (e.g. sensors), is used to perform automatic attention monitoring, helping to manage students in e-Learning. Additionally, the document presents a possible scenario which supports intelligent services to the monitoring of student attention during e-learning activities in the context of Smart HEI (Higher Education Institutes)

Air Force Institute of Technology Research Report 2010

This report summarizes the research activities of the Air Force Institute of Technology’s Graduate School of Engineering and Management. It describes research interests and faculty expertise; lists student theses/dissertations; identifies research sponsors and contributions; and outlines the procedures for contacting the school. Included in the report are: faculty publications, conference presentations, consultations, and funded research projects. Research was conducted in the areas of Aeronautical and Astronautical Engineering, Electrical Engineering and Electro-Optics, Computer Engineering and Computer Science, Systems and Engineering Management, Operational Sciences, Mathematics, Statistics and Engineering Physic

High Accuracy Distributed Target Detection and Classification in Sensor Networks Based on Mobile Agent Framework

High-accuracy distributed information exploitation plays an important role in sensor networks. This dissertation describes a mobile-agent-based framework for target detection and classification in sensor networks. Specifically, we tackle the challenging problems of multiple- target detection, high-fidelity target classification, and unknown-target identification. In this dissertation, we present a progressive multiple-target detection approach to estimate the number of targets sequentially and implement it using a mobile-agent framework. To further improve the performance, we present a cluster-based distributed approach where the estimated results from different clusters are fused. Experimental results show that the distributed scheme with the Bayesian fusion method have better performance in the sense that they have the highest detection probability and the most stable performance. In addition, the progressive intra-cluster estimation can reduce data transmission by 83.22% and conserve energy by 81.64% compared to the centralized scheme. For collaborative target classification, we develop a general purpose multi-modality, multi-sensor fusion hierarchy for information integration in sensor networks. The hierarchy is com- posed of four levels of enabling algorithms: local signal processing, temporal fusion, multi-modality fusion, and multi-sensor fusion using a mobile-agent-based framework. The fusion hierarchy ensures fault tolerance and thus generates robust results. In the meanwhile, it also takes into account energy efficiency. Experimental results based on two field demos show constant improvement of classification accuracy over different levels of the hierarchy. Unknown target identification in sensor networks corresponds to the capability of detecting targets without any a priori information, and of modifying the knowledge base dynamically. In this dissertation, we present a collaborative method to solve this problem among multiple sensors. When applied to the military vehicles data set collected in a field demo, about 80% unknown target samples can be recognized correctly, while the known target classification ac- curacy stays above 95%

Machine learning to model health with multimodal mobile sensor data

The widespread adoption of smartphones and wearables has led to the accumulation of rich datasets, which could aid the understanding of behavior and health in unprecedented detail. At the same time, machine learning and specifically deep learning have reached impressive performance in a variety of prediction tasks, but their use on time-series data appears challenging. Existing models struggle to learn from this unique type of data due to noise, sparsity, long-tailed distributions of behaviors, lack of labels, and multimodality. This dissertation addresses these challenges by developing new models that leverage multi-task learning for accurate forecasting, multimodal fusion for improved population subtyping, and self-supervision for learning generalized representations. We apply our proposed methods to challenging real-world tasks of predicting mental health and cardio-respiratory fitness through sensor data. First, we study the relationship of passive data as collected from smartphones (movement and background audio) to momentary mood levels. Our new training pipeline, which combines different sensor data into a low-dimensional embedding and clusters longitudinal user trajectories as outcome, outperforms traditional approaches based solely on psychology questionnaires. Second, motivated by mood instability as a predictor of poor mental health, we propose encoder-decoder models for time-series forecasting which exploit the bi-modality of mood with multi-task learning. Next, motivated by the success of general-purpose models in vision and language tasks, we propose a self-supervised neural network ready-to-use as a feature extractor for wearable data. To this end, we set the heart rate responses as the supervisory signal for activity data, leveraging their underlying physiological relationship and show that the resulting task-agnostic embeddings can generalize in predicting structurally different downstream outcomes through transfer learning (e.g. BMI, age, energy expenditure), outperforming unsupervised autoencoders and biomarkers. Finally, acknowledging fitness as a strong predictor of overall health, which, however, can only be measured with expensive instruments (e.g., a VO2max test), we develop models that enable accurate prediction of fine-grained fitness levels with wearables in the present, and more importantly, its direction and magnitude almost a decade later. All proposed methods are evaluated on large longitudinal datasets with tens of thousands of participants in the wild. The models developed and the insights drawn in this dissertation provide evidence for a better understanding of high-dimensional behavioral and physiological data with implications for large-scale health and lifestyle monitoring.The Department of Computer Science and Technology at the University of Cambridge through the EPSRC through Grant DTP (EP/N509620/1), and the Embiricos Trust Scholarship of Jesus College Cambridg

Detection and Prediction of Freezing of Gait in Parkinson’s Disease using Wearable Sensors and Machine Learning

Freezing of gait (FOG), is a brief episodic absence of forward body progression despite the intention to walk. Appearing mostly in mid-late stage Parkinson’s disease (PD), freezing manifests as a sudden loss of lower-limb function, and is closely linked to falling, decreased functional mobility, and loss of independence. Wearable-sensor based devices can detect freezes already in progress, and intervene by delivering auditory, visual, or tactile stimuli called cues. Cueing has been shown to reduce FOG duration and allow walking to continue. However, FOG detection and cueing systems require data from the freeze episode itself and are thus unable to prevent freezing. Anticipating the FOG episode before onset and supplying a timely cue could prevent the freeze from occurring altogether. FOG has been predicted in offline analyses by training machine learning models to identify wearable-sensor signal patterns known to precede FOG. The most commonly used sensors for FOG detection and prediction are inertial measurement units (IMU) that include an accelerometer, gyroscope and sometimes magnetometer. Currently, the best FOG prediction systems use data collected from multiple sensors on various body locations to develop person-specific models. Multi-sensor systems are more complex and may be challenging to integrate into real-life assistive devices. The ultimate goal of FOG prediction systems is a user-friendly assistive device that can be used by anyone experiencing FOG. To achieve this goal, person-independent models with high FOG prediction performance and a minimal number of conveniently located sensors are needed. The objectives of this thesis were: to develop and evaluate FOG detection and prediction models using IMU and plantar pressure data; determine if event-based or period of gait disruption FOG definitions have better classification performance for FOG detection and prediction; and evaluate FOG prediction models that use a single unilateral plantar pressure insole sensor or bilateral sensors. In this thesis, IMU (accelerometer and gyroscope) and plantar pressure insole sensors were used to collect data from 11 people with FOG while they walked a freeze provoking path. A custom-made synchronization and labeling program was used synchronize the IMU and plantar pressure data and annotate FOG episodes. Data were divided into overlapping 1 s windows with 0.2 s shift between consecutive windows. Time domain, Fourier transform based, and wavelet transform based features were extracted from the data. A total of 861 features were extracted from each of the 71,000 data windows. To evaluate the effectiveness of FOG detection and prediction models using plantar pressure and IMU data features, three feature sets were compared: plantar pressure, IMU, and both plantar pressure and IMU features. Minimum-redundancy maximum-relevance (mRMR) and Relief-F feature selection were performed prior to training boosted ensembles of decision trees. The binary classification models identified Total-FOG or Non-FOG states, wherein the Total-FOG class included windows with data from 2 s before the FOG onset until the end of the FOG episode. The plantar-pressure-only model had the greatest sensitivity, and the IMU-only model had the greatest specificity. The best overall model used the combination of plantar pressure and IMU features, achieving 76.4% sensitivity and 86.2% specificity. Next, the Total-FOG class components were evaluated individually (i.e., Pre-FOG windows, freeze windows, and transition windows between Pre-FOG and FOG). The best model, which used plantar pressure and IMU features, detected windows that contained both Pre-FOG and FOG data with 85.2% sensitivity, which is equivalent to detecting FOG less than 1 s after the freeze began. Models using both plantar pressure and IMU features performed better than models that used either sensor type alone. Datasets used to train machine learning models often generate ground truth FOG labels based on visual observation of specific lower limb movements (event-based definition) or an overall inability to walk effectively (period of gait disruption based definition). FOG definition ambiguity may affect FOG detection and prediction model performance, especially with respect to multiple FOG in rapid succession. This research examined the effects of defining FOG either as a period of gait disruption (merging successive FOG), or based on an event (no merging), on FOG detection and prediction. Plantar pressure and lower limb acceleration data were used to extract a set of features and train decision tree ensembles. FOG was labeled using an event-based definition. Additional datasets were then produced by merging FOG that occurred in rapid succession. A merging threshold was introduced where FOG that were separated by less than the merging threshold were merged into one episode. FOG detection and prediction models were trained for merging thresholds of 0, 1, 2, and 3 s. Merging had little effect on FOG detection model performance; however, for the prediction model, merging resulted in slightly later FOG identification and lower precision. FOG prediction models may benefit from using event-based FOG definitions and avoiding merging multiple FOG in rapid succession. Despite the known asymmetry of PD motor symptom manifestation, the difference between the more severely affected side (MSS) and less severely affected side (LSS) is rarely considered in FOG detection and prediction studies. The additional information provided by the MSS or LSS, if any, may be beneficial to FOG prediction models, especially if using a single sensor. To examine the effect of using data from the MSS, LSS, or both limbs, multiple FOG prediction models were trained and compared. Three datasets were created using plantar pressure data from the MSS, LSS, and both sides together. Feature selection was performed, and FOG prediction models were trained using the top 5, 10, 15, 20, 25 or 30 features for each dataset. The best models were the MSS model with 15 features, and the LSS and bilateral features with 5 features. The LSS model reached the highest sensitivity (79.5%) and identified the highest percentage of FOG episodes (94.9%). The MSS model achieved the highest specificity (84.9%) and the lowest false positive (FP) rate (2 FP/walking trial). Overall, the bilateral model was best. The bilateral model had 77.3% sensitivity, 82.9% specificity, and identified 94.3% of FOG episodes an average of 1.1 s before FOG onset. Compared to the bilateral model, the LSS model had a higher false positive rate; however, the bilateral and LSS models were similar in all other evaluation metrics. Therefore, using the LSS model instead of the bilateral model would produce similar FOG prediction performance at the cost of slightly more false positives. Given the advantages of single sensor systems, the increased FP rate may be acceptable. Therefore, a single plantar pressure sensor placed on the LSS could be used to develop a FOG prediction system and produce performance similar to a bilateral system

Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey

Time Series Classification and Extrinsic Regression are important and challenging machine learning tasks. Deep learning has revolutionized natural language processing and computer vision and holds great promise in other fields such as time series analysis where the relevant features must often be abstracted from the raw data but are not known a priori. This paper surveys the current state of the art in the fast-moving field of deep learning for time series classification and extrinsic regression. We review different network architectures and training methods used for these tasks and discuss the challenges and opportunities when applying deep learning to time series data. We also summarize two critical applications of time series classification and extrinsic regression, human activity recognition and satellite earth observation

Gait analysis in neurological populations: Progression in the use of wearables

Gait assessment is an essential tool for clinical applications not only to diagnose different neurological conditions but also to monitor disease progression as it contributes to the understanding of underlying deficits. There are established methods and models for data collection and interpretation of gait assessment within different pathologies. This narrative review aims to depict the evolution of gait assessment from observation and rating scales to wearable sensors and laboratory technologies, and provide possible future directions. In this context, we first present an extensive review of current clinical outcomes and gait models. Then, we demonstrate commercially available wearable technologies with their technical capabilities along with their use in gait assessment studies for various neurological conditions. In the next sections, a descriptive knowledge for existing inertial based algorithms and a sign based guide that shows the outcomes of previous neurological gait assessment studies are presented. Finally, we state a discussion for the use of wearables in gait assessment and speculate the possible research directions by revealing the limitations and knowledge gaps in the literature

Air Force Institute of Technology Research Report 2013

This report summarizes the research activities of the Air Force Institute of Technology’s Graduate School of Engineering and Management. It describes research interests and faculty expertise; lists student theses/dissertations; identifies research sponsors and contributions; and outlines the procedures for contacting the school. Included in the report are: faculty publications, conference presentations, consultations, and funded research projects. Research was conducted in the areas of Aeronautical and Astronautical Engineering, Electrical Engineering and Electro-Optics, Computer Engineering and Computer Science, Systems Engineering and Management, Operational Sciences, Mathematics, Statistics and Engineering Physics