Temporal convolutional networks for multi-person activity recognition using a 2D LIDAR

Motion trajectories contain rich information about human activities. We propose to use a 2D LIDAR to perform multiple people activity recognition simultaneously by classifying their trajectories. We clustered raw LIDAR data and classified the clusters into human and non-human classes in order to recognize humans in a scenario. For the clusters of humans, we implemented the Kalman Filter to track their trajectories which are further segmented and labelled with corresponding activities. We introduced spatial transformation and Gaussian noise for trajectory augmentation in order to overcome the problem of unbalanced classes and boost the performance of human activity recognition (HAR). Finally, we built two neural networks including a long short-term memory (LSTM) network and a temporal convolutional network (TCN) to classify trajectory samples into 15 activity classes collected from a kitchen. The proposed TCN achieved the best result of 99.49% in overall accuracy. In comparison, the TCN is slightly superior to the LSTM network. Both the TCN and the LSTM network outperform hidden Markov Model (HMM), dynamic time warping (DTW), and support vector machine (SVM) with a wide margin. Our approach achieves a higher activity recognition accuracy than the related work

Spatiotemporal graphical modeling for cyber-physical systems

Cyber-Physical Systems (CPSs) are combinations of physical processes and network computation. Modern CPSs such as smart buildings, power plants, transportation networks, and power-grids have shown tremendous potential for increased efficiency, robustness, and resilience. However, such modern CPSs encounter a large variety of physical faults and cyber anomalies, and in many cases are vulnerable to catastrophic fault propagation scenarios due to strong connectivity among their sub-systems. To address these issues, this study proposes a graphical modeling framework to monitor and predict the performance of CPSs in a scalable and robust way. This thesis investigates on two critical CPS applications to evaluate the effectiveness of this proposed framework, namely (i) health monitoring of highway traffic sensors and (ii) building energy consumption prediction. In highway traffic sensor networks, accurate traffic sensor data is essential for traffic operation management systems and acquisition of real-time traffic surveillance data depends heavily on the reliability of the physical systems. Therefore, detecting the health status of the sensors in a traffic sensor network is critical for the departments of transportation as well as other public and private entities, especially in the circumstances where real-time decision making is required. With the purpose of efficiently determining the traffic network status and identifying failed sensor(s), this study proposes a cost-effective spatiotemporal graphical modeling approach called spatiotemporal pattern network (STPN). Traffic speed and volume measurement sensors are used in this work to formulate and analyze the proposed sensor health monitoring system. The historical time-series data from the networked traffic sensors on the Interstate 35 (I-35) within the state of Iowa is used for validation. Based on the validation results, this study demonstrates that the proposed graphical modeling approach can: (i) extract spatiotemporal dependencies among the different sensors which lead to an efficient graphical representation of the sensor network in the information space, and (ii) distinguish and quantify a sensor issue by leveraging the extracted spatiotemporal relationship of the candidate sensor(s) to the other sensors in the network. In the building energy consumption prediction case, we consider the fact that energy performance of buildings is primarily affected by the heat exchange with the building outer skin and the surrounding environment. In addition, it is a common practice in building energy simulation (BES) to predict energy usage with a variable degree of accuracy. Therefore, to account for accurate building energy consumption, especially in urban environments with a lot of anthropogenic heat sources, it is necessary to consider the microclimate conditions around the building. These conditions are influenced by the immediate environment, such as surrounding buildings, hard surfaces, and trees. Moreover, deployment of sensors to monitor the microclimate information of a building can be quite challenging and therefore, not scalable. Instead of applying local weather data directly on building energy simulation (BES) tools, this work proposes a spatiotemporal pattern network (STPN) based machine learning framework to predict the microclimate information based on the local weather station, which leads to better energy consumption prediction in buildings

Latitude, longitude, and beyond:mining mobile objects' behavior

Rapid advancements in Micro-Electro-Mechanical Systems (MEMS), and wireless communications, have resulted in a surge in data generation. Mobility data is one of the various forms of data, which are ubiquitously collected by different location sensing devices. Extensive knowledge about the behavior of humans and wildlife is buried in raw mobility data. This knowledge can be used for realizing numerous viable applications ranging from wildlife movement analysis, to various location-based recommendation systems, urban planning, and disaster relief. With respect to what mentioned above, in this thesis, we mainly focus on providing data analytics for understanding the behavior and interaction of mobile entities (humans and animals). To this end, the main research question to be addressed is: How can behaviors and interactions of mobile entities be determined from mobility data acquired by (mobile) wireless sensor nodes in an accurate and efficient manner? To answer the above-mentioned question, both application requirements and technological constraints are considered in this thesis. On the one hand, applications requirements call for accurate data analytics to uncover hidden information about individual behavior and social interaction of mobile entities, and to deal with the uncertainties in mobility data. Technological constraints, on the other hand, require these data analytics to be efficient in terms of their energy consumption and to have low memory footprint, and processing complexity

Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Visualizing the Motion Flow of Crowds

In modern cities, massive population causes problems, like congestion, accident, violence and crime everywhere. Video surveillance system such as closed-circuit television cameras is widely used by security guards to monitor human behaviors and activities to manage, direct, or protect people. With the quantity and prolonged duration of the recorded videos, it requires a huge amount of human resources to examine these video recordings and keep track of activities and events. In recent years, new techniques in computer vision field reduce the barrier of entry, allowing developers to experiment more with intelligent surveillance video system. Different from previous research, this dissertation does not address any algorithm design concerns related to object detection or object tracking. This study will put efforts on the technological side and executing methodologies in data visualization to find the model of detecting anomalies. It would like to provide an understanding of how to detect the behavior of the pedestrians in the video and find out anomalies or abnormal cases by using techniques of data visualization

Big Data Computing for Geospatial Applications

The convergence of big data and geospatial computing has brought forth challenges and opportunities to Geographic Information Science with regard to geospatial data management, processing, analysis, modeling, and visualization. This book highlights recent advancements in integrating new computing approaches, spatial methods, and data management strategies to tackle geospatial big data challenges and meanwhile demonstrates opportunities for using big data for geospatial applications. Crucial to the advancements highlighted in this book is the integration of computational thinking and spatial thinking and the transformation of abstract ideas and models to concrete data structures and algorithms

Statistical and Machine Learning Models for Remote Sensing Data Mining - Recent Advancements

This book is a reprint of the Special Issue entitled "Statistical and Machine Learning Models for Remote Sensing Data Mining - Recent Advancements" that was published in Remote Sensing, MDPI. It provides insights into both core technical challenges and some selected critical applications of satellite remote sensing image analytics