Mask Estimation For Missing Data Recognition Using Background Noise Sniffing

This paper addresses the problem of spectrographic mask estimation in the context of missing data recognition. At the difference of other denoising methods, missing data recognition does not match the whole spectrum with the acoustic models, but rather considers that some time-frequency pixels are missing, i.e. corrupted by noise. Correctly estimating these ``masks'' is very important for missing data recognizers. We propose a new approach that exploits some a priori knowledge about these masks in typical noisy environments to address this difficult challenge. The proposed mask is then obtained by combining these noise dependent masks. The combination is led by an environmental ``sniffing'' module that estimates the probability of being in each typical noisy condition. This missing data mask estimation procedure has been integrated in a complete missing data recognizer using bounded marginalization. Our approach is evaluated on the Aurora2 database

Missing data mask models with global frequency and temporal constraints

Missing data recognition has been developped in order to increase noise robustness in automatic speech recognition. Many different factors, including the speech decoding process itself, shall be considered to locate the masks. In this work, we are considering Bayesian models of the masks, where every spectral feature is classified as reliable or masked, and is independent from the rest of the signal. This classification strategy can produce unrelated small ``spots'', while experiments suggest that oracle reliable and unreliable features tend to be clustered into time-frequency blocks. We call this undesired effect: the ``checkerboard'' effect. In this paper, we propose a new Bayesian missing data classifier that integrates frequency and temporal constraints in order to reduce, or avoid, this ``checkerboard'' effect. The proposed classifier is evaluated on the Aurora2 connected digit corpora. Integrating such constraints in the missing data classification leads to significant improvements in recognition accuracy

A Supervised Approach to Global Signal-to-Noise Ratio Estimation for Whispered and Pathological Voices

The presence of background noise in signals adversely affects the performance of many speech-based algorithms. Accurate estimation of signal-to-noise-ratio (SNR), as a measure of noise level in a signal, can help in compensating for noise effects. Most existing SNR estimation methods have been developed for normal speech and might not provide accurate estimation for special speech types such as whispered or disordered voices, particularly, when they are corrupted by non-stationary noises. In this paper, we first investigate the impact of stationary and non-stationary noise on the behavior of mel-frequency cepstral coefficients (MFCCs) extracted from normal, whispered and pathological voices. We demonstrate that, regardless of the speech type, the mean and the covariance of MFCCs are predictably modified by additive noise and the amount of change is related to the noise level. Then, we propose a new supervised method for SNR estimation which is based on a regression model trained on MFCCs of the noisy signals. Experimental results show that the proposed approach provides accurate estimation and consistent performance for various speech types under different noise conditions

Disentangling rodent behaviors to improve automated behavior recognition

Automated observation and analysis of behavior is important to facilitate progress in many fields of science. Recent developments in deep learning have enabled progress in object detection and tracking, but rodent behavior recognition struggles to exceed 75–80% accuracy for ethologically relevant behaviors. We investigate the main reasons why and distinguish three aspects of behavior dynamics that are difficult to automate. We isolate these aspects in an artificial dataset and reproduce effects with the state-of-the-art behavior recognition models. Having an endless amount of labeled training data with minimal input noise and representative dynamics will enable research to optimize behavior recognition architectures and get closer to human-like recognition performance for behaviors with challenging dynamics

The Chemical Senses

Long-standing neglect of the chemical senses in the philosophy of perception is due, mostly, to their being regarded as ‘lower’ senses. Smell, taste, and chemically irritated touch are thought to produce mere bodily sensations. However, empirically informed theories of perception can show how these senses lead to perception of objective properties, and why they cannot be treated as special cases of perception modelled on vision. The senses of taste, touch, and smell also combine to create unified perceptions of flavour. The nature of these multimodal experiences and the character of our awareness of them puts pressure on the traditional idea that each episode of perception goes one or other of the five senses. Thus, the chemical senses, far from being peripheral to the concerns of the philosophy of perception, may hold important clues to the multisensory nature of perception in general

Neuropsychology and neuroimaging in diffuse brain damage : a study of visual event perception

The aims of this project were (1) to investigate two forms of event perception: perception of movement and perception of sudden appearance, (2) to develop event perception procedures which could be applied to testing clinical populations, and (3) to relate event perception to abnormalities shown by neuroimaging. In addition issues relevant to each of the particular clinical populations involved were addressed. Event perception tasks used stimuli consisting of a background of randomly selected dots of light. In one task a dot was added to the display (appearance), in the other a dot started to move (movement onset). Four laboratory experiments were conducted examining the ability to detect and locate these events under varying conditions in healthy controls. Results indicated that neuronal coding strategies were different for appearances and movement onset. Laboratory tasks were adapted for clinical application and administered to groups of patients with different neurological conditions. Five studies were conducted to assess sensitivity and specificity of the Event Perception tasks in clinical settings. The groups studied were chronic solvent abusers, detoxified alcoholics, patients suffering from optic neuritis, and patients with traumatic brain injury. Event Perception tasks were found to be differentially sensitive to neurological conditions and showed dissociations and double dissociations both within and between neurological conditions. Relationships with Magnetic Resonance Imaging (MRI) and Single Photon Emission Computed Tomography (SPECT) were investigated in patients with head injury. Patterns of brain damage differed significantly for patients with impaired performance on the movement task. It is concluded that Event Perception tasks are of value in the assessment of neurological patients: They allow assessment of functions which are not usually evaluated in neuropsychological examinations, facilitate detection of subtle deficits and deficits which may present at an early stage, and offer greater specificity and sensitivity than many traditional neuropsychological test procedures. Event Perception tasks are easy to administer and do not suffer from training effects on repeated administration to the same degree as many traditional measures. It is also argued that tests with a theoretical basis are better suited to clinical research in neuropsychology than many traditional tasks because they potentially allow a more precise explanation and assessment of the abnormal processes under investigation

An object's smell in the multisensory brain : how our senses interact during olfactory object processing

Object perception is a remarkable and fundamental cognitive ability that allows us to interpret and interact with the world we are living in. In our everyday life, we constantly perceive objects–mostly without being aware of it and through several senses at the same time. Although it might seem that object perception is accomplished without any effort, the underlying neural mechanisms are anything but simple. How we perceive objects in the world surrounding us is the result of a complex interplay of our senses. The aim of the present thesis was to explore, by means of functional magnetic resonance imaging, how our senses interact when we perceive an object’s smell in a multisensory setting where the amount of sensory stimulation increases, as well as in a unisensory setting where we perceive an object’s smell in isolation. In Study I, we sought to determine whether and how multisensory object information influences the processing of olfactory object information in the posterior piriform cortex (PPC), a region linked to olfactory object encoding. In Study II, we then expanded our search for integration effects during multisensory object perception to the whole brain because previous research has demonstrated that multisensory integration is accomplished by a network of early sensory cortices and higher-order multisensory integration sites. We specifically aimed at determining whether there exist cortical regions that process multisensory object information independent of from which senses and from how many senses the information arises. In Study III, we then sought to unveil how our senses interact during olfactory object perception in a unisensory setting. Other previous studies have shown that even in such unisensory settings, olfactory object processing is not exclusively accomplished by regions within the olfactory system but instead engages a more widespread network of brain regions, such as regions belonging to the visual system. We aimed at determining what this visual engagement represents. That is, whether areas of the brain that are principally concerned with processing visual object information also hold neural representations of olfactory object information, and if so, whether these representations are similar for smells and pictures of the same objects. In Study I we demonstrated that assisting inputs from our senses of vision and hearing increase the processing of olfactory object information in the PPC, and that the more assisting input we receive the more the processing is enhanced. As this enhancement occurred only for matching inputs, it likely reflects integration of multisensory object information. Study II provided evidence for convergence of multisensory object information in form of a non-linear response enhancement in the inferior parietal cortex: activation increased for bimodal compared to unimodal stimulation, and increased even further for trimodal compared to bimodal stimulation. As this multisensory response enhancement occurred independent of the congruency of the incoming signals, it likely reflects a process of relating the incoming sensory information streams to each other. Finally, Study III revealed that regions of the ventral visual object stream are engaged in recognition of an object’s smell and represent olfactory object information in form of distinct neural activation patterns. While the visual system encodes information about both visual and olfactory objects, it appears to keep information from the two sensory modalities separate by representing smells and pictures of objects differently. Taken together, the studies included in this thesis reveal that olfactory object perception is a multisensory process that engages a widespread network of early sensory as well higher-order cortical regions, even if we do not encounter ourselves in a multisensory setting but exclusively perceive an object’s smell

Sniffing out Parkinson's disease : Psychophysical and neurophysiological studies of impaired olfactory information processing in Parkinson's disease

Wolters, E.C.H. [Promotor]Stam, C.J. [Promotor]Berendse, H.W. [Copromotor

Machine Learning Based Detection of False Data Injection Attacks in Wide Area Monitoring Systems

The Smart Grid (SG) is an upgraded, intelligent, and a more reliable version of the traditional Power Grid due to the integration of information and communication technologies. The operation of the SG requires a dense communication network to link all its components. But such a network renders it prone to cyber attacks jeopardizing the integrity and security of the communicated data between the physical electric grid and the control centers. One of the most prominent components of the SG are Wide Area Monitoring Systems (WAMS). WAMS are a modern platform for grid-wide information, communication, and coordination that play a major role in maintaining the stability of the grid against major disturbances. In this thesis, an anomaly detection framework is proposed to identify False Data Injection (FDI) attacks in WAMS using different Machine Learning (ML) and Deep Learning (DL) techniques, i.e., Deep Autoencoders (DAE), Long-Short Term Memory (LSTM), and One-Class Support Vector Machine (OC-SVM). These algorithms leverage diverse, complex, and high-volume power measurements coming from communications between different components of the grid to detect intelligent FDI attacks. The injected false data is assumed to target several major WAMS monitoring applications, such as Voltage Stability Monitoring (VSM), and Phase Angle Monitoring (PAM). The attack vector is considered to be smartly crafted based on the power system data, so that it can pass the conventional bad data detection schemes and remain stealthy. Due to the lack of realistic attack data, machine learning-based anomaly detection techniques are used to detect FDI attacks. To demonstrate the impact of attacks on the realistic WAMS traffic and to show the effectiveness of the proposed detection framework, a Hardware-In-the-Loop (HIL) co-simulation testbed is developed. The performance of the implemented techniques is compared on the testbed data using different metrics: Accuracy, F1 score, and False Positive Rate (FPR) and False Negative Rate (FNR). The IEEE 9-bus and IEEE 39-bus systems are used as benchmarks to investigate the framework scalability. The experimental results prove the effectiveness of the proposed models in detecting FDI attacks in WAMS

Peek-a-Boo: I see your smart home activities, even encrypted!

A myriad of IoT devices such as bulbs, switches, speakers in a smart home environment allow users to easily control the physical world around them and facilitate their living styles through the sensors already embedded in these devices. Sensor data contains a lot of sensitive information about the user and devices. However, an attacker inside or near a smart home environment can potentially exploit the innate wireless medium used by these devices to exfiltrate sensitive information from the encrypted payload (i.e., sensor data) about the users and their activities, invading user privacy. With this in mind,in this work, we introduce a novel multi-stage privacy attack against user privacy in a smart environment. It is realized utilizing state-of-the-art machine-learning approaches for detecting and identifying the types of IoT devices, their states, and ongoing user activities in a cascading style by only passively sniffing the network traffic from smart home devices and sensors. The attack effectively works on both encrypted and unencrypted communications. We evaluate the efficiency of the attack with real measurements from an extensive set of popular off-the-shelf smart home IoT devices utilizing a set of diverse network protocols like WiFi, ZigBee, and BLE. Our results show that an adversary passively sniffing the traffic can achieve very high accuracy (above 90%) in identifying the state and actions of targeted smart home devices and their users. To protect against this privacy leakage, we also propose a countermeasure based on generating spoofed traffic to hide the device states and demonstrate that it provides better protection than existing solutions.Comment: Update (May 13, 2020): This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in the 13th ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec '20), July 8-10, 2020, Linz (Virtual Event), Austria, https://doi.org/10.1145/3395351.339942