Human Localization and Activity Recognition Using Distributed Motion Sensors

The purpose of this thesis is to localize a human and recognize his/her activities in indoor environments using distributed motion sensors. We propose to use a test bed simulated as mock apartment for conducting our experiments. The two parts of the thesis are localization and activity recognition of the elderly person. We explain complete hardware and software setup used to provide these services. The hardware setup consists of two types of sensor end nodes and two sink nodes. The two types of end nodes are Passive Infrared sensor node and GridEye sensor node. Passive Infrared sensor nodes consist of Passive Infrared sensors for motion detection. GridEye sensor nodes consist of thermal array sensors. Data from these sensors are acquired using Arduino boards and transmitted using Xbee modules to the sink nodes. The sink nodes consist of receiver Xbee modules connected to a computer. The sensor nodes were strategically placed at different place inside the apartment. The thermal array sensor provides 64 pixel temperature values, while the PIR sensor provides binary information about motion in its field of view. Since the thermal array sensor provides more information, they were placed in large rooms such as living room and bed room. While PIR sensors were placed in kitchen and bathroom. Initially GridEye sensors are calibrated to obtain the transformation between pixel and real world coordinates. Data from these sensors were processed on computer and we were able to localize the human inside the apartment. We compared the location accuracy using ground truth data obtained from the OptiTrack system. GridEye sensors were also used for activity recognition. Basic human activities such as sitting, sleeping, standing and walking were recognized. We used Support Vector Machine (SVM) to recognize sitting and sleeping activities. Gait speed of human was used to recognize the standing and walking activities. Experiments were performed to obtain the accuracy of classification for these activities.Electrical Engineerin

Detection of Human by Thermopile Infrared Sensors

13301甲第4416号博士(工学)金沢大学博士論文本文Ful

Indoor Human Detection Based on Thermal Array Sensor Data and Adaptive Background Estimation

Low Resolution Thermal Array Sensors are widely used in several applications in indoor environments. In particular, one of these cheap, small and unobtrusive sensors provides a low-resolution thermal image of the environment and, unlike cameras; it is capable to detect human heat emission even in dark rooms. The obtained thermal data can be used to monitor older seniors while they are performing daily activities at home, to detect critical situations such as falls. Most of the studies in activity recognition using Thermal Array Sensors require human detection techniques to recognize humans passing in the sensor field of view. This paper aims to improve the accuracy of the algorithms used so far by considering the temperature environment variation. This method leverages an adaptive background estimation and a noise removal technique based on Kalman Filter. In order to properly validate the system, a novel installation of a single sensor has been implemented in a smart environment: the obtained results show an improvement in human detection accuracy with respect to the state of the art, especially in case of disturbed environments

Multiple thermal sensor array fusion towards enabling privacy-preserving human monitoring applications

Human-centric applications of a single Thermal Sensor Array (TSA) have performed extremely well in many areas. However, most of these works have not yet reached the real applicability stage of the Internet of Things (IoT) applications. The main limitation of deploying such systems on a large scale is the challenge of fusing multiple TSAs to cover a wide inspection area, e.g. smart homes, hospitals and many other domestic environments. On the other hand, objects that appear in the low-resolution thermal images acquired from TSA have low intra-class variations and high inter-class similarities, making the identification of the overlapping regions through matching a comparable template image in multiple images very difficult. This paper proposes a motion-based approach to fuse multiple TSAs and learn the domestic environment layout to enable further human-centred IoT applications to run in the cloud. Besides, a privacy-improvement on utilising these sensors in IoT applications is proposed. The proposed approach is evaluated with comprehensive experiments on different sensor placements and domestic environment conditions. This paper shows an average performance of 92.5% accuracy using various machine learning techniques and use case scenarios

Indoor location based services challenges, requirements and usability of current solutions

Indoor Location Based Services (LBS), such as indoor navigation and tracking, still have to deal with both technical and non-technical challenges. For this reason, they have not yet found a prominent position in people’s everyday lives. Reliability and availability of indoor positioning technologies, the availability of up-to-date indoor maps, and privacy concerns associated with location data are some of the biggest challenges to their development. If these challenges were solved, or at least minimized, there would be more penetration into the user market. This paper studies the requirements of LBS applications, through a survey conducted by the authors, identifies the current challenges of indoor LBS, and reviews the available solutions that address the most important challenge, that of providing seamless indoor/outdoor positioning. The paper also looks at the potential of emerging solutions and the technologies that may help to handle this challenge

Device-free indoor localisation with non-wireless sensing techniques : a thesis by publications presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Electronics and Computer Engineering, Massey University, Albany, New Zealand

Global Navigation Satellite Systems provide accurate and reliable outdoor positioning to support a large number of applications across many sectors. Unfortunately, such systems do not operate reliably inside buildings due to the signal degradation caused by the absence of a clear line of sight with the satellites. The past two decades have therefore seen intensive research into the development of Indoor Positioning System (IPS). While considerable progress has been made in the indoor localisation discipline, there is still no widely adopted solution. The proliferation of Internet of Things (IoT) devices within the modern built environment provides an opportunity to localise human subjects by utilising such ubiquitous networked devices. This thesis presents the development, implementation and evaluation of several passive indoor positioning systems using ambient Visible Light Positioning (VLP), capacitive-flooring, and thermopile sensors (low-resolution thermal cameras). These systems position the human subject in a device-free manner (i.e., the subject is not required to be instrumented). The developed systems improve upon the state-of-the-art solutions by offering superior position accuracy whilst also using more robust and generalised test setups. The developed passive VLP system is one of the first reported solutions making use of ambient light to position a moving human subject. The capacitive-floor based system improves upon the accuracy of existing flooring solutions as well as demonstrates the potential for automated fall detection. The system also requires very little calibration, i.e., variations of the environment or subject have very little impact upon it. The thermopile positioning system is also shown to be robust to changes in the environment and subjects. Improvements are made over the current literature by testing across multiple environments and subjects whilst using a robust ground truth system. Finally, advanced machine learning methods were implemented and benchmarked against a thermopile dataset which has been made available for other researchers to use

Activity Recognition in Residential Spaces with Internet of Things Devices and Thermal Imaging

In this paper, we design algorithms for indoor activity recognition and 3D thermal model generation using thermal images, RGB images, captured from external sensors, and the internet of things setup. Indoor activity recognition deals with two sub-problems: Human activity and household activity recognition. Household activity recognition includes the recognition of electrical appliances and their heat radiation with the help of thermal images. A FLIR ONE PRO camera is used to capture RGB-thermal image pairs for a scene. Duration and pattern of activities are also determined using an iterative algorithm, to explore kitchen safety situations. For more accurate monitoring of hazardous events such as stove gas leakage, a 3D reconstruction approach is proposed to determine the temperature of all points in the 3D space of a scene. The 3D thermal model is obtained using the stereo RGB and thermal images for a particular scene. Accurate results are observed for activity detection, and a significant improvement in the temperature estimation is recorded in the 3D thermal model compared to the 2D thermal image. Results from this research can find applications in home automation, heat automation in smart homes, and energy management in residential spaces

Privacy-preserving human behaviour monitoring through thermal vision

Despite the abundance of human-centred research to support domestic human behaviour monitoring in various vital applications, there are still notable limitations to deploying such systems on a broader scale. The main challenge is the trade-off between privacy, performance, and cost of assistive technologies to support older adults to live independently in their own homes. For example, the traditional vision-based sensing approach provides excellent performance while violating human privacy in domestic environments. In contrast, the ambient sensing approach, e.g., employing Passive Infra-Red (PIR) sensors, maintains human privacy but suffers significant performance hindrances in realistic scenarios such as multi-occupancy environments. This research proposes to utilise the Thermal Sensor Array (TSA) to adjust the trade-off between privacy and performance in domestic environment applications. The rationale behind proposing this sensor for human behaviour monitoring applications is its claimed advantages to perform well while maintaining human privacy, low-cost, and noncontact capabilities. Nevertheless, there has not been sufficient related work to empirically validate the hypothesis of using this low-resolution imager in domestic monitoring. Furthermore, most published works that use the TSA have not yet reached the deployment stage due to the TSA sensing constraints. In particular, TSA is sensitive to environmental thermal noise, and its Field of View (FoV) is not wide enough to cover a large inspection area. Intelligent algorithms should be employed in order to avoid these limitations. The focus of this thesis is to investigate the human physiological and behavioural thermal patterns for privacy-preserving human behaviour monitoring to support the independent living of older adults in a multi-occupancy environment by using TSA. This will be achieved through signal processing and machine learning techniques. To achieve this aim, the research methodology is drawn into two main directions. First, human physiological processing of the human thermal signal. Second, human behavioural processing of the human motion signal. This drawn methodology resulted in four main novel contributions. The first novel contribution of this research is to propose an adaptive segmentation of the human physiological presence and count the number of people from different sensor placements, indoor environments, and human-to-sensor distance. The second contribution is to extract localisation knowledge of the human physiological appearance in terms of human-to-sensor distance and human-to-human distance. Extracting human localisation knowledge is also applicable in other applications such as caregivers and care time monitoring. The third contribution is to fuse multiple TSAs to cover a wide inspection area, e.g., private or care homes. Hence, objects that appear in the low-resolution thermal images acquired from TSA have low intra-class variations and high inter-class similarities, making the identification of the overlapping regions through matching a comparable template image in multiple images very difficult. This research proposes a motion-based approach to fuse multiple TSAs and learn the domestic environment layout with a privacy improvement of utilising TSA in potential monitoring applications running in the cloud. Inspired by the results from this stage of the research, the fourth contribution of the research presented in this thesis is a human-in-the-loop fall detection approach in the Activities of Daily Living (ADLs) that reduces the false-positive alerts while keeping the false-negative fall predictions as low as possible. The novel solutions and the results presented in this thesis demonstrate a significant contribution toward enabling privacy-preserving human behaviour monitoring

Wearable Biosensors to Understand Construction Workers' Mental and Physical Stress

Occupational stress is defined as harmful physical and mental responses when job requirements are greater than a worker's capacity. Construction is one of the most stressful occupations because it involves physiologically and psychologically demanding tasks performed in a hazardous environment this stress can jeopardize construction safety, health, and productivity. Various instruments, such as surveys and interviews, have been used for measuring workers’ perceived mental and physical stress. However valuable, such instruments are limited by their invasiveness, which prevents them from being used for continuous stress monitoring. The recent advancement of wearable biosensors has opened a new door toward the non-invasive collection of a field worker’s physiological signals that can be used to assess their mental and physical status. Despite these advancements, challenges remain: acquiring physiological signals from wearable biosensors can be easily contaminated from diverse sources of signal noise. Further, the potential of these devices to assess field workers’ mental and physical status has not been examined in the naturalistic work environment. To address these issues, this research aims to propose and validate a comprehensive and efficient stress-measurement framework that recognizes workers mental and physical stress in a naturalistic environment. The focus of this research is on two wearable biosensors. First, a wearable EEG headset, which is a direct measurement of brain waves with the minimal time lag, but it is highly vulnerable to various artifacts. Second, a very convenient wristband-type biosensor, which may be used as a means for assessing both mental and physical stress, but there is a time lag between when subjects are exposed to stressors and when their physiological signals change. To achieve this goal, five interrelated and interdisciplinary studies were performed to; 1) acquire high-quality EEG signals from the job site; 2) assess construction workers’ emotion by measuring the valence and arousal level by analyzing the patterns of construction workers’ brainwaves; 3) recognize mental stress in the field based on brain activities by applying supervised-learning algorithms;4) recognize real-time mental stress by applying Online Multi-Task Learning (OMTL) algorithms; and 5) assess workers’ mental and physical stress using signals collected from a wristband biosensor. To examine the performance of the proposed framework, we collected physiological signals from 21 workers at five job sites. Results yielded a high of 80.13% mental stress-recognition accuracy using an EEG headset and 90.00% physical stress-recognition accuracy using a wristband sensor. These results are promising given that stress recognition with wired physiological devices within a controlled lab setting in the clinical domain has, at best, a similar level of accuracy. The proposed wearable biosensor-based, stress-recognition framework is expected to help us better understand workplace stressors and improve worker safety, health, and productivity through early detection and mitigation of stress at human-centered, smart and connected construction sites.PHDCivil EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/149965/1/hjebelli_1.pd