シュリーレン法による可聴音場可視化のための時空間フィルタリング

早大学位記番号:新7470早稲田大

A Mobile App Illustrating Sensory Neural Coding Through an Efficient Coding of Collected Images and Sounds

Sensory neuroscience in the early auditory and visual systems appears distinct not only to outside observers, but to many trained neuroscientists as well. However, to a computational neuroscientist, both sensory systems represent an efficient neural coding of information. In fact, on a computational level it appears the brain is using the same processing strategy for both senses - the same algorithm with just a change in inputs. Insights like this can greatly simplify our understanding of the brain, but require a significant computational background to fully appreciate. How can such illuminating results of computational neuroscience be made more accessible to the entire neuroscience community? We built an Android mobile app that simulates the neural coding process in the early visual and auditory system. The app demonstrates the type of visual or auditory codes that would develop depending on the images or sounds that an evolving species would be exposed to over evolutionary time. This is done by visually displaying the derived image and sound filters based on an optimal encoding that information, and comparing them to visual representations of neural receptive fields in the brain. Image patches (or equivalently, sound clips) are efficiently encoded using Independent Components Analysis (ICA) as a proxy for the coding objective of the early visual system. As has been observed for the past two decades, the resulting code from natural images resembles the 2D Gabor filter receptive fields measured from neurons in primary visual cortex (V1). Similarly, this efficient encoding demonstration has been done for a mixture of natural sounds to create linear filters resembling the gammatone filters of the spiral ganglia from the cochlea. The app demonstrates the relationship between efficient codes of images and sounds and related sensory neural coding in an intuitive, accessible way. This enables budding neuroscientists, and even the general public, to appreciate how an understanding of computational tools (like ICA or sparse coding) can bridge research across seemingly distinct areas of the brain. This enables a more parsimonious view of how the brain processes information, and may encourage early-program neuroscientists to consider improving their computational skills

Fall Detection Using Neural Networks

Falls inside of the home is a major concern facing the aging population. Monitoring the home environment to detect a fall can prevent profound consequences due to delayed emergency response. One option to monitor a home environment is to use a camera-based fall detection system. Conceptual designs vary from 3D positional monitoring (multi-camera monitoring) to body position and limb speed classification. Research shows varying degree of success with such concepts when designed with multi-camera setup. However, camera-based systems are inherently intrusive and costly to implement. In this research, we use a sound-based system to detect fall events. Acoustic sensors are used to monitor various sound events and feed a trained machine learning model that makes predictions of a fall events. Audio samples from the sensors are converted to frequency domain images using Mel-Frequency Cepstral Coefficients method. These images are used by a trained convolution neural network to predict a fall. A publicly available dataset of household sounds is used to train the model. Varying the model\u27s complexity, we found an optimal architecture that achieves high performance while being computationally less extensive compared to the other models with similar performance. We deployed this model in a NVIDIA Jetson Nano Developer Kit

Use of Pattern Classification Algorithms to Interpret Passive and Active Data Streams from a Walking-Speed Robotic Sensor Platform

In order to perform useful tasks for us, robots must have the ability to notice, recognize, and respond to objects and events in their environment. This requires the acquisition and synthesis of information from a variety of sensors. Here we investigate the performance of a number of sensor modalities in an unstructured outdoor environment, including the Microsoft Kinect, thermal infrared camera, and coffee can radar. Special attention is given to acoustic echolocation measurements of approaching vehicles, where an acoustic parametric array propagates an audible signal to the oncoming target and the Kinect microphone array records the reflected backscattered signal. Although useful information about the target is hidden inside the noisy time domain measurements, the Dynamic Wavelet Fingerprint process (DWFP) is used to create a time-frequency representation of the data. A small-dimensional feature vector is created for each measurement using an intelligent feature selection process for use in statistical pattern classification routines. Using our experimentally measured data from real vehicles at 50 m, this process is able to correctly classify vehicles into one of five classes with 94% accuracy. Fully three-dimensional simulations allow us to study the nonlinear beam propagation and interaction with real-world targets to improve classification results

2D and 3D visualization of acoustic waves by optical feedback interferometry

The visualization of physical phenomena is one of the challenges that researchers are trying to overcome by designing and implementing different sensors that provide information close to realitythrough changes in one of the parameters they measure. Historically, the visualization of variations in physical phenomena has allowed for a better understanding of the problem being studied and has changed our perception of the world and ourselves forever. Over the last 300 years, in particular, many methods have been developed to visualize sound through a visual representation. In the field of acoustics, scientists have attempted to develop a visual representation of sound waves using transducers detecting two fundamental components of sound: sound pressure and particle velocity. In other words, the measurement of kinetic energy and potential, whose quantities provide information on the physical phenomenon of acoustic propagation. In this summary, we briefly present the work of the thesis entitled "2D and 3D Visualizations of Acoustic Waves by Optical Feedback Interferometry" in which a new visualization tool for acoustic phenomena was developed. This system is based on an optical sensor said reinjection in a laser diode and allows to reconstruct in 2D and 3D the image of a propagating acoustic wave. The manuscript is divided into 3 chapters: • a first chapter presents the known methods for the visualization of the acoustic phenomena and presents the context of the research carried out, • a second chapter, allows to detail the principle of measurement and its application to the realization of a two-dimensional image of the acoustic wave • finally, in the last chapter, we demonstrate how a tomographic method can be used to create a three-dimensional image

Bio-Acoustic Tracking and Localization Using Heterogeneous, Scalable Microphone Arrays

Microphone arrays are an essential tool in the field of bioacoustics as they provide a non-intrusive way to study animal vocalizations and monitor their movement and behavior. Microphone arrays can be used for passive localization and tracking of sound sources while analyzing beamforming or spatial filtering of the emitted sound. Studying free roaming animals usually requires setting up equipment over large areas and attaching a tracking device to the animal which may alter their behavior. However, monitoring vocalizing animals through arrays of microphones, spatially distributed over their habitat has the advantage that unrestricted/unmanipulated animals can be observed. Important insights have been achieved through the use of microphone arrays, such as the convergent acoustic field of view in echolocating bats or context-dependent functions of avian duets. Here we show the development and application of large flexible microphone arrays that can be used to localize and track any vocalizing animal and study their bio-acoustic behavior. In a first experiment with hunting pallid bats the acoustic data acquired from a dense array with 64 microphones revealed details of the bats’ echolocation beam in previously unseen resolution. We also demonstrate the flexibility of the proposed microphone array system in a second experiment, where we used a different array architecture allowing to simultaneously localize several species of vocalizing songbirds in a radius of 75 m. Our technology makes it possible to do longer measurement campaigns over larger areas studying changing habitats and providing new insights for habitat conservation. The flexible nature of the technology also makes it possible to create dense microphone arrays that can enhance our understanding in various fields of bioacoustics and can help to tackle the analytics of complex behaviors of vocalizing animals

Sound Processing for Autonomous Driving

Nowadays, a variety of intelligent systems for autonomous driving have been developed, which have already shown a very high level of capability. One of the prerequisites for autonomous driving is an accurate and reliable representation of the environment around the vehicle. Current systems rely on cameras, RADAR, and LiDAR to capture the visual environment and to locate and track other traffic participants. Human drivers, in addition to vision, have hearing and use a lot of auditory information to understand the environment in addition to visual cues. In this thesis, we present the sound signal processing system for auditory based environment representation. Sound propagation is less dependent on occlusion than all other types of sensors and in some situations is less sensitive to different types of weather conditions such as snow, ice, fog or rain. Various audio processing algorithms provide the detection and classification of different audio signals specific to certain types of vehicles, as well as localization. First, the ambient sound is classified into fourteen major categories consisting of traffic objects and actions performed. Additionally, the classification of three specific types of emergency vehicles sirens is provided. Secondly, each object is localized using a combined localization algorithm based on time difference of arrival and amplitude. The system is evaluated on real data with a focus on reliable detection and accurate localization of emergency vehicles. On the third stage the possibility of visualizing the sound source on the image from the autonomous vehicle camera system is provided. For this purpose, a method for camera to microphones calibration has been developed. The presented approaches and methods have great potential to increase the accuracy of environment perception and, consequently, to improve the reliability and safety of autonomous driving systems in general