A survey on artificial intelligence-based acoustic source identification

The concept of Acoustic Source Identification (ASI), which refers to the process of identifying noise sources has attracted increasing attention in recent years. The ASI technology can be used for surveillance, monitoring, and maintenance applications in a wide range of sectors, such as defence, manufacturing, healthcare, and agriculture. Acoustic signature analysis and pattern recognition remain the core technologies for noise source identification. Manual identification of acoustic signatures, however, has become increasingly challenging as dataset sizes grow. As a result, the use of Artificial Intelligence (AI) techniques for identifying noise sources has become increasingly relevant and useful. In this paper, we provide a comprehensive review of AI-based acoustic source identification techniques. We analyze the strengths and weaknesses of AI-based ASI processes and associated methods proposed by researchers in the literature. Additionally, we did a detailed survey of ASI applications in machinery, underwater applications, environment/event source recognition, healthcare, and other fields. We also highlight relevant research directions

In-situ health monitoring for wind turbine blade using acoustic wireless sensor networks at low sampling rates

PhD ThesisThe development of in-situ structural health monitoring (SHM) techniques represents a challenge for offshore wind turbines (OWTs) in order to reduce the cost of the operation and maintenance (O&M) of safety-critical components and systems. This thesis propos- es an in-situ wireless SHM system based on acoustic emission (AE) techniques. The proposed wireless system of AE sensor networks is not without its own challenges amongst which are requirements of high sampling rates, limitations in the communication bandwidth, memory space, and power resources. This work is part of the HEMOW- FP7 Project, ‘The Health Monitoring of Offshore Wind Farms’. The present study investigates solutions relevant to the abovementioned challenges. Two related topics have been considered: to implement a novel in-situ wireless SHM technique for wind turbine blades (WTBs); and to develop an appropriate signal pro- cessing algorithm to detect, localise, and classify different AE events. The major contri- butions of this study can be summarised as follows: 1) investigating the possibility of employing low sampling rates lower than the Nyquist rate in the data acquisition opera- tion and content-based feature (envelope and time-frequency data analysis) for data analysis; 2) proposing techniques to overcome drawbacks associated with lowering sampling rates, such as information loss and low spatial resolution; 3) showing that the time-frequency domain is an effective domain for analysing the aliased signals, and an envelope-based wavelet transform cross-correlation algorithm, developed in the course of this study, can enhance the estimation accuracy of wireless acoustic source localisa- tion; 4) investigating the implementation of a novel in-situ wireless SHM technique with field deployment on the WTB structure, and developing a constraint model and approaches for localisation of AE sources and environmental monitoring respectively. Finally, the system has been experimentally evaluated with the consideration of the lo- calisation and classification of different AE events as well as changes of environmental conditions. The study concludes that the in-situ wireless SHM platform developed in the course of this research represents a promising technique for reliable SHM for OWTBs in which solutions for major challenges, e.g., employing low sampling rates lower than the Nyquist rate in the acquisition operation and resource constraints of WSNs in terms of communication bandwidth and memory space are presente

WiFi Sensing at the Edge Towards Scalable On-Device Wireless Sensing Systems

WiFi sensing offers a powerful method for tracking physical activities using the radio-frequency signals already found throughout our homes and offices. This novel sensing modality offers continuous and non-intrusive activity tracking since sensing can be performed (i) without requiring wearable sensors, (ii) outside the line-of-sight, and even (iii) through the wall. Furthermore, WiFi has become a ubiquitous technology in our computers, our smartphones, and even in low-cost Internet of Things devices. In this work, we consider how the ubiquity of these low-cost WiFi devices offer an unparalleled opportunity for improving the scalability of wireless sensing systems. Thus far, WiFi sensing research assumes costly offline computing resources and hardware for training machine learning models and for performing model inference. To improve the scalability of WiFi sensing systems, this dissertation introduces techniques for improving machine learning at the edge by thoroughly surveying and evaluating signal preprocessing and edge machine learning techniques. Additionally, we introduce the use of federated learning for collaboratively training machine learning models with WiFi data only available on edge devices. We then consider privacy and security concerns of WiFi sensing by demonstrating possible adversarial surveillance attacks. To combat these attacks, we propose a method for leveraging spatially distributed antennas to prevent eavesdroppers from performing adversarial surveillance while still enabling and even improving the sensing capabilities of allowed WiFi sensing devices within our environments. The overall goal throughout this work is to demonstrate that WiFi sensing can become a ubiquitous and secure sensing option through the use of on-device computation on low-cost edge devices

Decision-Making with Heterogeneous Sensors - A Copula Based Approach

Statistical decision making has wide ranging applications, from communications and signal processing to econometrics and finance. In contrast to the classical one source-one receiver paradigm, several applications have been identified in the recent past that require acquiring data from multiple sources or sensors. Information from the multiple sensors are transmitted to a remotely located receiver known as the fusion center which makes a global decision. Past work has largely focused on fusion of information from homogeneous sensors. This dissertation extends the formulation to the case when the local sensors may possess disparate sensing modalities. Both the theoretical and practical aspects of multimodal signal processing are considered. The first and foremost challenge is to \u27adequately\u27 model the joint statistics of such heterogeneous sensors. We propose the use of copula theory for this purpose. Copula models are general descriptors of dependence. They provide a way to characterize the nonlinear functional relationships between the multiple modalities, which are otherwise difficult to formalize. The important problem of selecting the `best\u27 copula function from a given set of valid copula densities is addressed, especially in the context of binary hypothesis testing problems. Both, the training-testing paradigm, where a training set is assumed to be available for learning the copula models prior to system deployment, as well as generalized likelihood ratio test (GLRT) based fusion rule for the online selection and estimation of copula parameters are considered. The developed theory is corroborated with extensive computer simulations as well as results on real-world data. Sensor observations (or features extracted thereof) are most often quantized before their transmission to the fusion center for bandwidth and power conservation. A detection scheme is proposed for this problem assuming unifom scalar quantizers at each sensor. The designed rule is applicable for both binary and multibit local sensor decisions. An alternative suboptimal but computationally efficient fusion rule is also designed which involves injecting a deliberate disturbance to the local sensor decisions before fusion. The rule is based on Widrow\u27s statistical theory of quantization. Addition of controlled noise helps to \u27linearize\u27 the higly nonlinear quantization process thus resulting in computational savings. It is shown that although the introduction of external noise does cause a reduction in the received signal to noise ratio, the proposed approach can be highly accurate when the input signals have bandlimited characteristic functions, and the number of quantization levels is large. The problem of quantifying neural synchrony using copula functions is also investigated. It has been widely accepted that multiple simultaneously recorded electroencephalographic signals exhibit nonlinear and non-Gaussian statistics. While the existing and popular measures such as correlation coefficient, corr-entropy coefficient, coh-entropy and mutual information are limited to being bivariate and hence applicable only to pairs of channels, measures such as Granger causality, even though multivariate, fail to account for any nonlinear inter-channel dependence. The application of copula theory helps alleviate both these limitations. The problem of distinguishing patients with mild cognitive impairment from the age-matched control subjects is also considered. Results show that the copula derived synchrony measures when used in conjunction with other synchrony measures improve the detection of Alzheimer\u27s disease onset

Machine Learning for Human Activity Detection in Smart Homes

Recognizing human activities in domestic environments from audio and active power consumption sensors is a challenging task since on the one hand, environmental sound signals are multi-source, heterogeneous, and varying in time and on the other hand, the active power consumption varies significantly for similar type electrical appliances. Many systems have been proposed to process environmental sound signals for event detection in ambient assisted living applications. Typically, these systems use feature extraction, selection, and classification. However, despite major advances, several important questions remain unanswered, especially in real-world settings. A part of this thesis contributes to the body of knowledge in the field by addressing the following problems for ambient sounds recorded in various real-world kitchen environments: 1) which features, and which classifiers are most suitable in the presence of background noise? 2) what is the effect of signal duration on recognition accuracy? 3) how do the SNR and the distance between the microphone and the audio source affect the recognition accuracy in an environment in which the system was not trained? We show that for systems that use traditional classifiers, it is beneficial to combine gammatone frequency cepstral coefficients and discrete wavelet transform coefficients and to use a gradient boosting classifier. For systems based on deep learning, we consider 1D and 2D CNN using mel-spectrogram energies and mel-spectrograms images, as inputs, respectively and show that the 2D CNN outperforms the 1D CNN. We obtained competitive classification results for two such systems and validated the performance of our algorithms on public datasets (Google Brain/TensorFlow Speech Recognition Challenge and the 2017 Detection and Classification of Acoustic Scenes and Events Challenge). Regarding the problem of the energy-based human activity recognition in a household environment, machine learning techniques to infer the state of household appliances from their energy consumption data are applied and rule-based scenarios that exploit these states to detect human activity are used. Since most activities within a house are related with the operation of an electrical appliance, this unimodal approach has a significant advantage using inexpensive smart plugs and smart meters for each appliance. This part of the thesis proposes the use of unobtrusive and easy-install tools (smart plugs) for data collection and a decision engine that combines energy signal classification using dominant classifiers (compared in advanced with grid search) and a probabilistic measure for appliance usage. It helps preserving the privacy of the resident, since all the activities are stored in a local database. DNNs received great research interest in the field of computer vision. In this thesis we adapted different architectures for the problem of human activity recognition. We analyze the quality of the extracted features, and more specifically how model architectures and parameters affect the ability of the automatically extracted features from DNNs to separate activity classes in the final feature space. Additionally, the architectures that we applied for our main problem were also applied to text classification in which we consider the input text as an image and apply 2D CNNs to learn the local and global semantics of the sentences from the variations of the visual patterns of words. This work helps as a first step of creating a dialogue agent that would not require any natural language preprocessing. Finally, since in many domestic environments human speech is present with other environmental sounds, we developed a Convolutional Recurrent Neural Network, to separate the sound sources and applied novel post-processing filters, in order to have an end-to-end noise robust system. Our algorithm ranked first in the Apollo-11 Fearless Steps Challenge.Horizon 2020 research and innovation program under the Marie Sklodowska-Curie grant agreement No. 676157, project ACROSSIN