Neural Networks for Indoor Person Tracking With Infrared Sensors

Indoor localization has many pervasive applications, like energy management, health monitoring, and security. Tagless localization detects directly the human body, like passive infrared sensing, and is the most amenable to different users and use cases. We evaluate the localization and tracking performance, as well as resource and processing requirements of various neural network (NN) types using directly the data from a low resolution 16-pixel thermopile sensor array in a 3 m x 3 m room. Out of the multilayer perceptron, autoregressive, 1D-CNN, and LSTM NN architectures that we test, the latter require more resources but can accurately locate and capture best the person movement dynamics, while the 1D-CNN provides the best compromise between localization accuracy (9.6 cm RMSE) and movement tracking smoothness with the least resources, and seem more suited for embedded applications

Device-free indoor localisation with non-wireless sensing techniques : a thesis by publications presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Electronics and Computer Engineering, Massey University, Albany, New Zealand

Global Navigation Satellite Systems provide accurate and reliable outdoor positioning to support a large number of applications across many sectors. Unfortunately, such systems do not operate reliably inside buildings due to the signal degradation caused by the absence of a clear line of sight with the satellites. The past two decades have therefore seen intensive research into the development of Indoor Positioning System (IPS). While considerable progress has been made in the indoor localisation discipline, there is still no widely adopted solution. The proliferation of Internet of Things (IoT) devices within the modern built environment provides an opportunity to localise human subjects by utilising such ubiquitous networked devices. This thesis presents the development, implementation and evaluation of several passive indoor positioning systems using ambient Visible Light Positioning (VLP), capacitive-flooring, and thermopile sensors (low-resolution thermal cameras). These systems position the human subject in a device-free manner (i.e., the subject is not required to be instrumented). The developed systems improve upon the state-of-the-art solutions by offering superior position accuracy whilst also using more robust and generalised test setups. The developed passive VLP system is one of the first reported solutions making use of ambient light to position a moving human subject. The capacitive-floor based system improves upon the accuracy of existing flooring solutions as well as demonstrates the potential for automated fall detection. The system also requires very little calibration, i.e., variations of the environment or subject have very little impact upon it. The thermopile positioning system is also shown to be robust to changes in the environment and subjects. Improvements are made over the current literature by testing across multiple environments and subjects whilst using a robust ground truth system. Finally, advanced machine learning methods were implemented and benchmarked against a thermopile dataset which has been made available for other researchers to use

Data Processing for Device-Free Fine-Grained Occupancy Sensing Using Infrared Sensors

Fine-grained occupancy information plays an essential role for various emerging applications in smart homes, such as personalized thermal comfort control and human behavior analysis. Existing occupancy sensors, such as passive infrared (PIR) sensors generally provide limited coarse information such as motion. However, the detection of fine-grained occupancy information such as stationary presence, posture, identification, and activity tracking can be enabled with the advance of sensor technologies. Among these, infrared sensing is a low-cost, device-free, and privacy-preserving choice that detects the fluctuation (PIR sensors) or the thermal profiles (thermopile array sensors) from objects' infrared radiation. This work focuses on developing data processing models towards fine-grained occupancy sensing using the synchronized low-energy electronically chopped PIR (SLEEPIR) sensor or the thermopile array sensors. The main contributions of this dissertation include: (1) creating and validating the mathematical model of the SLEEPIR sensor output towards stationary occupancy detection; (2) developing the SLEEPIR detection algorithm using statistical features and long-short term memory (LSTM) deep learning; (3) building machine learning framework for posture detection and activity tracking using thermopile array sensors; and (4) creating convolutional neural network (CNN) models for facing direction detection and identification using thermopile array sensors

Lab-on-a-chip Thermoelectric and Solid-phase Immunodetection of Biochemical Analytes and Extracellular Vesicles: Experimental and Computational Analysis

Microfluidics is the technology of controlling and manipulating fluids at the microscale. Microfluidic platforms provide precise fluidic control coupled with low sample volume and an increase in the speed of biochemical reactions. Lab-on-a-chip platforms are used for detection and quantification of biochemical analytes, capture, and characterization of various proteins, sensitive analysis of cytokines, and isolation and detection of extracellular vesicles (EVs). This study focuses on the development of microfluidic and solid-phase capture pin platforms for the detection of cytokines, extracellular vesicles, and cell co-culture. The fabrication processes of the devices, experimental workflows, numerical analysis to identify optimal design parameters, and reproducibility studies have been discussed. Layer-by-layer assembly of polyelectrolytes has been developed to functionalize glass and stainless-steel substrates with biotin for the immobilization of streptavidinconjugated antibodies for selective capture of cytokines or EVs. Microstructure characterization techniques (SEM, EDX, and fluorescence microscopy) have been implemented to assess the efficiency of substrate functionalization. A detailed overview of current methods for purification and analysis of EVs is discussed as well. Additionally, the dissertation demonstrates the feasibility of a calorimetric microfluidic immunosensor with an integrated antimony-bismuth (Sb/Bi) thermopile sensor for the detection of cytokines with picomolar sensitivity. The developed platform can be used for the universal detection of both exothermic or endothermic reactions. A three-dimensional numerical model was developed to define the critical design parameters that enhance the sensitivity of the platform. Mathematical analyses identified the optimal combinations of substrate material and dimensions that will maximize the heat transfer to the sensor. Lab-on-a-chip cell co-culture platform with integrated pneumatic valve was designed, numerically characterized, and fabricated. This device enables the reversible separation of two cell culture chambers and serves as a tool for the effective analysis of cell-to-cell communication. Intercellular communication is mediated by extracellular vesicles. A protocol for the functionalization of stainless-steel probe with exosomespecific CD63 antibody was developed. The efficiency of the layer-by-layer deposition of polyelectrolytes and the effectiveness of biotin and streptavidin covalent boding were characterized using fluorescent and scanning electron microscopy

Design of a rodent dynamometer

In the past, computer controlled dynamometers have been used to perform isometric and isovelocity testing to study the effects of velocity, acceleration, and position on muscle injury. However, the effect of constant force on muscle injury in animals has not yet been studied.;A computer-controlled dynamometer was designed which could perform isometric, isovelocity, and isotonic testing on both male and female rats. A mechanical device was designed to position the rats for testing as well as provide a platform for the motor and other system components. All of the system components were connected electrically through a National Instruments UMI-7764 breakout box, which allowed for convenient integration of all the components. Individual programs were written in LabVIEW to perform concentric and eccentric isometric, isovelocity, and isotonic testing. The programs allowed for many variations to the standard programming such as isometric holds, repetitive cycles, and rest times

Privacy-preserving human behaviour monitoring through thermal vision

Despite the abundance of human-centred research to support domestic human behaviour monitoring in various vital applications, there are still notable limitations to deploying such systems on a broader scale. The main challenge is the trade-off between privacy, performance, and cost of assistive technologies to support older adults to live independently in their own homes. For example, the traditional vision-based sensing approach provides excellent performance while violating human privacy in domestic environments. In contrast, the ambient sensing approach, e.g., employing Passive Infra-Red (PIR) sensors, maintains human privacy but suffers significant performance hindrances in realistic scenarios such as multi-occupancy environments. This research proposes to utilise the Thermal Sensor Array (TSA) to adjust the trade-off between privacy and performance in domestic environment applications. The rationale behind proposing this sensor for human behaviour monitoring applications is its claimed advantages to perform well while maintaining human privacy, low-cost, and noncontact capabilities. Nevertheless, there has not been sufficient related work to empirically validate the hypothesis of using this low-resolution imager in domestic monitoring. Furthermore, most published works that use the TSA have not yet reached the deployment stage due to the TSA sensing constraints. In particular, TSA is sensitive to environmental thermal noise, and its Field of View (FoV) is not wide enough to cover a large inspection area. Intelligent algorithms should be employed in order to avoid these limitations. The focus of this thesis is to investigate the human physiological and behavioural thermal patterns for privacy-preserving human behaviour monitoring to support the independent living of older adults in a multi-occupancy environment by using TSA. This will be achieved through signal processing and machine learning techniques. To achieve this aim, the research methodology is drawn into two main directions. First, human physiological processing of the human thermal signal. Second, human behavioural processing of the human motion signal. This drawn methodology resulted in four main novel contributions. The first novel contribution of this research is to propose an adaptive segmentation of the human physiological presence and count the number of people from different sensor placements, indoor environments, and human-to-sensor distance. The second contribution is to extract localisation knowledge of the human physiological appearance in terms of human-to-sensor distance and human-to-human distance. Extracting human localisation knowledge is also applicable in other applications such as caregivers and care time monitoring. The third contribution is to fuse multiple TSAs to cover a wide inspection area, e.g., private or care homes. Hence, objects that appear in the low-resolution thermal images acquired from TSA have low intra-class variations and high inter-class similarities, making the identification of the overlapping regions through matching a comparable template image in multiple images very difficult. This research proposes a motion-based approach to fuse multiple TSAs and learn the domestic environment layout with a privacy improvement of utilising TSA in potential monitoring applications running in the cloud. Inspired by the results from this stage of the research, the fourth contribution of the research presented in this thesis is a human-in-the-loop fall detection approach in the Activities of Daily Living (ADLs) that reduces the false-positive alerts while keeping the false-negative fall predictions as low as possible. The novel solutions and the results presented in this thesis demonstrate a significant contribution toward enabling privacy-preserving human behaviour monitoring

Proceedings of the Scientific-Practical Conference "Research and Development - 2016"

talent management; sensor arrays; automatic speech recognition; dry separation technology; oil production; oil waste; laser technolog

Home healthcare using ubiquitous computing and robot technologies

The rapid increase of senior population worldwide is challenging the existing healthcare and support systems. Recently, smart home environments are utilized for ubiquitous health monitoring, allowing patients to stay in the comfort of their homes. In this dissertation, a Cloud-based Smart Home Environment (CoSHE) for home healthcare is presented, which consists of ambient intelligence, wearable computing, and robot technologies. The system includes a smart home which is embedded with distributed environmental sensors to support human localization. Wearable units are developed to collect physiological, motion and audio signals through non-invasive wearable sensors and provide contextual information in terms of the resident's daily activity and location in the home. This enables healthcare professionals to study daily activities, behavioral changes and monitor rehabilitation and recovery processes. The sensor data are processed in a smart home gateway and sent to a private cloud, which provides real-time data access for remote caregivers. Our case studies show that contextual information provided by ubiquitous computing can help better understand the patient's health status. With a robot assistant in the loop, we demonstrated that the CoSHE can facilitate healthcare delivery via interaction between human and robot

MEMS-based temperature-dependent characterization of biomolecular interactions

Biomolecular interactions are of fundamental importance for a wide variety of biological processes. Temperature dependence is a ubiquitous effect for biomolecular interactions as most biological processes are thermally active. Understanding the temperature dependence of biomolecular interactions is hence critical for a wide variety of applications in fundamental sciences and drug discovery and biotherapeutics. Micro-Electro-Mechanical Systems (MEMS) technology holds great potential in facilitating temperature-dependent characterization of biomolecular interactions by providing on-chip microfluidic handling with drastically reduced sample consumption, and well-controlled micro- or nanoscale environments in which biomolecules are effectively manipulated and analyzed. This thesis is focused on various MEMS-based devices for temperature-dependent characterization of biomolecular interactions. Biomolecular interactions can occur with biomolecules in solution or with either the target or receptor molecules immobilized to a solid surface. For surface-based biomolecular interactions, we first present microcantilever-based characterization of biomolecular affinity binding with in-situ temperature sensing, using a demonstrative system of platelet-derived growth factor (PDGF) and an inhibitory ligand. The temperature-dependent kinetic and equilibrium binding properties are determined. In addition, a microfluidic approach for temperature-dependent biomolecular behavior with single-molecule resolution is also presented. Using a platform that combines microfludic sample handling, on-chip temperature control, and total internal reflection fluorescence (TIRF) microscopy, we have studied the temperature dependence of the structural dynamics of transfer RNA (tRNA) translocation through ribosome in protein synthesis. For solution-based biomolecular interactions, we mainly focus on calorimetry, a technology that directly measures heat evolved in biological processes. We first present a MEMS differential scanning calorimetric (DSC) sensor integrating highly sensitive thermoelectric sensing and microfluidic handling for thermodynamic characterization of biomolecules. We have characterized the unfolding of protein (e.g. lysozyme) at minimized sample consumption with thermodynamic properties determined, including the specific heat capacity, molar enthalpy change, and melting temperature. In addition, we also present the development of a variant of standard DSC, temperature-modulated DSC (AC-DSC), on a MEMS device for thermodynamic characterization of biomolecules. Preliminary results again with lysozyme unfolding at optimum modulation frequencies have been presented with thermodynamic properties determined. Furthermore, we have developed a MEMS isothermal titration calorimeter (ITC) integrating thermally isolated calorimetric chambers, on-chip passive mixing, and environmental temperature control, for temperature-dependent characterization of biomolecular interactions. We have characterized the interactions of 18-Crown-6 and barium chloride, as well as ribonuclease A and cytidine 2'-monophosphate, in a 1-µL volume with low concentrations (ca. 2 mM). Thermodynamic properties, including the stoichiometry, equilibrium binding constant, and enthalpy change, are also determined