A Home Security System Based on Smartphone Sensors

Several new smartphones are released every year. Many people upgrade to new phones, and their old phones are not put to any further use. In this paper, we explore the feasibility of using such retired smartphones and their on-board sensors to build a home security system. We observe that door-related events such as opening and closing have unique vibration signatures when compared to many types of environmental vibrational noise. These events can be captured by the accelerometer of a smartphone when the phone is mounted on a wall near a door. The rotation of a door can also be captured by the magnetometer of a smartphone when the phone is mounted on a door. We design machine learning and threshold-based methods to detect door opening events based on accelerometer and magnetometer data and build a prototype home security system that can detect door openings and notify the homeowner via email, SMS and phone calls upon break-in detection. To further augment our security system, we explore using the smartphone’s built-in microphone to detect door and window openings across multiple doors and windows simultaneously. Experiments in a residential home show that the accelerometer- based detection can detect door open events with an accuracy higher than 98%, and magnetometer-based detection has 100% accuracy. By using the magnetometer method to automate the training phase of a neural network, we find that sound-based detection of door openings has an accuracy of 90% across multiple doors

SPEAKER AND GENDER IDENTIFICATION USING BIOACOUSTIC DATA SETS

Acoustic analysis of animal vocalizations has been widely used to identify the presence of individual species, classify vocalizations, identify individuals, and determine gender. In this work automatic identification of speaker and gender of mice from ultrasonic vocalizations and speaker identification of meerkats from their Close calls is investigated. Feature extraction was implemented using Greenwood Function Cepstral Coefficients (GFCC), designed exclusively for extracting features from animal vocalizations. Mice ultrasonic vocalizations were analyzed using Gaussian Mixture Models (GMM) which yielded an accuracy of 78.3% for speaker identification and 93.2% for gender identification. Meerkat speaker identification with Close calls was implemented using Gaussian Mixture Models (GMM) and Hidden Markov Models (HMM), with an accuracy of 90.8% and 94.4% respectively. The results obtained shows these methods indicate the presence of gender and identity information in vocalizations and support the possibility of robust gender identification and individual identification using bioacoustic data sets

ARTICULATORY INFORMATION FOR ROBUST SPEECH RECOGNITION

Current Automatic Speech Recognition (ASR) systems fail to perform nearly as good as human speech recognition performance due to their lack of robustness against speech variability and noise contamination. The goal of this dissertation is to investigate these critical robustness issues, put forth different ways to address them and finally present an ASR architecture based upon these robustness criteria. Acoustic variations adversely affect the performance of current phone-based ASR systems, in which speech is modeled as `beads-on-a-string', where the beads are the individual phone units. While phone units are distinctive in cognitive domain, they are varying in the physical domain and their variation occurs due to a combination of factors including speech style, speaking rate etc.; a phenomenon commonly known as `coarticulation'. Traditional ASR systems address such coarticulatory variations by using contextualized phone-units such as triphones. Articulatory phonology accounts for coarticulatory variations by modeling speech as a constellation of constricting actions known as articulatory gestures. In such a framework, speech variations such as coarticulation and lenition are accounted for by gestural overlap in time and gestural reduction in space. To realize a gesture-based ASR system, articulatory gestures have to be inferred from the acoustic signal. At the initial stage of this research an initial study was performed using synthetically generated speech to obtain a proof-of-concept that articulatory gestures can indeed be recognized from the speech signal. It was observed that having vocal tract constriction trajectories (TVs) as intermediate representation facilitated the gesture recognition task from the speech signal. Presently no natural speech database contains articulatory gesture annotation; hence an automated iterative time-warping architecture is proposed that can annotate any natural speech database with articulatory gestures and TVs. Two natural speech databases: X-ray microbeam and Aurora-2 were annotated, where the former was used to train a TV-estimator and the latter was used to train a Dynamic Bayesian Network (DBN) based ASR architecture. The DBN architecture used two sets of observation: (a) acoustic features in the form of mel-frequency cepstral coefficients (MFCCs) and (b) TVs (estimated from the acoustic speech signal). In this setup the articulatory gestures were modeled as hidden random variables, hence eliminating the necessity for explicit gesture recognition. Word recognition results using the DBN architecture indicate that articulatory representations not only can help to account for coarticulatory variations but can also significantly improve the noise robustness of ASR system

Fall Detection Using Channel State Information from WiFi Devices

Falls among the independently living elderly population are a major public health worry, leading to injuries, loss of confidence to live independently and even to death. Each year, one in three people aged 65 and older falls and one in five of them suffers fatal or non fatal injuries. Therefore, detecting a fall early and alerting caregivers can potentially save lives and increase the standard of living. Existing solutions, e.g. push-button, wearables, cameras, radar, pressure and vibration sensors, have limited public adoption either due to the requirement for wearing the device at all times or installing specialized and expensive infrastructure. In this thesis, a device-free, low cost indoor fall detection system using commodity WiFi devices is presented. The system uses physical layer Channel State Information (CSI) to detect falls. Commercial WiFi hardware is cheap and ubiquitous and CSI provides a wealth of information which helps in maintaining good fall detection accuracy even in challenging environments. The goals of the research in this thesis are the design, implementation and experimentation of a device-free fall detection system using CSI extracted from commercial WiFi devices. To achieve these objectives, the following contributions are made herein. A novel time domain human presence detection scheme is developed as a precursor to detecting falls. As the next contribution, a novel fall detection system is designed and developed. Finally, two main enhancements to the fall detection system are proposed to improve the resilience to changes in operating environment. Experiments were performed to validate system performance in diverse environments. It can be argued that through collection of real world CSI traces, understanding the behavior of CSI during human motion, the development of a signal processing tool-set to facilitate the recognition of falls and validation of the system using real world experiments significantly advances the state of the art by providing a more robust fall detection scheme

Securing Additive Manufacturing Systems from Cyber and Intellectual Property Attacks

Additive Manufacturing (AM), also known as 3D printing, refers to a collection of manufacturing processes where materials are joined together layer by layer to make objects directly from 3D models. Due to many advantages of AM, such as rapid prototyping, massive customization, material saving, and flexibility of designs, there is a trend for AM to replace traditional manufacturing processes. However, AM highly relies on computers to work. As AM systems are gaining popularity in many critical industry sectors, there is an increased risk of cyberattacks on AM systems. To protect AM systems from cyberattacks that aim to sabotage the AM systems, Intrusion Detection Systems (IDSs) can be used. In recent years, researchers proposed a series of IDSs that work by leveraging side-channel signals. A side-channel signal is typically a physical signal that is correlated with the state of the AM system, such as the acoustic sound or the electromagnetic wave emitted by a 3D printer in a printing process. Because of the correlation between a side-channel signal and the state of a 3D printer, it is possible to perform intrusion detection by analyzing the side-channel signal. In fact, most existing IDSs leveraging side-channel signals in AM systems function by comparing an observed side-channel signal against a reference side-channel signal. However, we found that these IDSs are not practical due to a lack of synchronization. Many IDSs in the literature do not contain details on how to align two (or more) side-channel signals at their starting moments and their stopping moments. In addition, we found that there is time noise in AM processes. When the same G-code file is executed on the same 3D printer multiple times, the printing processes will have slightly different timing. Because of time noise, a direct comparison between two signals point by point or window by window will not make sense. To overcome this problem, we propose to use dynamic synchronization to find corresponding points between two signals in real time. To demonstrate the necessity of dynamic synchronization, we performed a total of 302 benign printing processes and a total of 200 malicious printing processes with two printers. Our experiment results show that existing IDSs leveraging side-channel signals in AM systems can only achieve an accuracy from 0.50 to 0.88, whereas our IDS with dynamic synchronization can reach an accuracy of 0.99. Other than cyberattacks to sabotage AM systems, there are also cyberattacks to steal intellectual property in AM systems. For example, there are acoustic side-channel attacks on AM systems which can recover the printing path by analyzing the acoustic sound by a printer in a printing process. However, we found that the acoustic side-channel attack is hard to perform due to challenges such as integration drift and non-unique solution. In this thesis, we explore the optical side-channel attack, which is much easier to perform than the acoustic side-channel attack. The optical side-channel signal is basically the video of a printing process. We use a modified deep neural network, ResNet50, to recognize the coordinates of the printhead in each frame in the video. To defend against the optical side-channel attack, we propose the optical noise injection method. We use an optical projector to artificially inject crafted optical noise onto the printing area in an attempt to confuse the attacker and make it harder to recover the printing path. We found that existing noise generation algorithms, such as replaying, random blobs, white noise, and full power, can effortlessly defeat a naive attacker who is not aware of the existence of the injected noise. However, an advanced attacker who knows about the injected noise and incorporates images with injected noise in the training dataset can defeat all of the existing noise generation algorithms. To defend against such an advanced attacker, we propose three novel noise generation algorithms: channel uniformization, state uniformization, and state randomization. Our experiment results show that noise generated via state randomization can successfully defeat the advanced attacker.Ph.D

Smartphone-Based Evaluation of Postural Stability in Parkinson’s Disease Patients During Quiet Stance

Background: Postural instability is one of the most troublesome motor symptoms of Parkinson’s Disease(PD).It impairs patients’quality of life and results in high risk of falls. The aim of this study is to provide a reliable tool for the automated assessment of postural instability. Methods: Data acquisition was performed on 42 PD patients and 7 young healthy subjects. They were asked to keep a quiet stance position for at least 30 s while wearing a waist-mounted smartphone. A total number of 414 features was extracted from both time and frequency domain, selected based on Pearson’s correlation, and fed to an optimized Support Vector Machine. Results: The implemented model was able to differentiate patients with mild postural instability from those with severe postural instability and from healthy controls, with 100% accuracy. Conclusion: This study demonstrated the feasibility of using inertial sensors embedded in commercial smartphones and proposed a simple protocol for accurate postural instability scoring. This tool can be used for early detection of PD motor signs, disease follow-up and fall prevention

ChoreoGraph: Music-conditioned Automatic Dance Choreography over a Style and Tempo Consistent Dynamic Graph

To generate dance that temporally and aesthetically matches the music is a challenging problem, as the following factors need to be considered. First, the aesthetic styles and messages conveyed by the motion and music should be consistent. Second, the beats of the generated motion should be locally aligned to the musical features. And finally, basic choreomusical rules should be observed, and the motion generated should be diverse. To address these challenges, we propose ChoreoGraph, which choreographs high-quality dance motion for a given piece of music over a Dynamic Graph. A data-driven learning strategy is proposed to evaluate the aesthetic style and rhythmic connections between music and motion in a progressively learned cross-modality embedding space. The motion sequences will be beats-aligned based on the music segments and then incorporated as nodes of a Dynamic Motion Graph. Compatibility factors such as the style and tempo consistency, motion context connection, action completeness, and transition smoothness are comprehensively evaluated to determine the node transition in the graph. We demonstrate that our repertoire-based framework can generate motions with aesthetic consistency and robustly extensible in diversity. Both quantitative and qualitative experiment results show that our proposed model outperforms other baseline models

Radar and RGB-depth sensors for fall detection: a review

This paper reviews recent works in the literature on the use of systems based on radar and RGB-Depth (RGB-D) sensors for fall detection, and discusses outstanding research challenges and trends related to this research field. Systems to detect reliably fall events and promptly alert carers and first responders have gained significant interest in the past few years in order to address the societal issue of an increasing number of elderly people living alone, with the associated risk of them falling and the consequences in terms of health treatments, reduced well-being, and costs. The interest in radar and RGB-D sensors is related to their capability to enable contactless and non-intrusive monitoring, which is an advantage for practical deployment and users’ acceptance and compliance, compared with other sensor technologies, such as video-cameras, or wearables. Furthermore, the possibility of combining and fusing information from The heterogeneous types of sensors is expected to improve the overall performance of practical fall detection systems. Researchers from different fields can benefit from multidisciplinary knowledge and awareness of the latest developments in radar and RGB-D sensors that this paper is discussing