Water sound recognition based on physical models

International audienceThis article describes an audio signal processing algorithm to detect water sounds, built in the context of a larger system aiming to monitor daily activities of elderly people. While previous proposals for water sound recognition relied on classical machine learning and generic audio features to characterize water sounds as a flow texture, we describe here a recognition system based on a physical model of air bubble acoustics. This system is able to recognize a wide variety of water sounds and does not require training. It is validated on a home environmental sound corpus with a classification task, in which all water sounds are correctly detected. In a free detection task on a real life recording, it outperformed the classical systems and obtained 70% of F-measure

Visually Indicated Sounds

Objects make distinctive sounds when they are hit or scratched. These sounds reveal aspects of an object's material properties, as well as the actions that produced them. In this paper, we propose the task of predicting what sound an object makes when struck as a way of studying physical interactions within a visual scene. We present an algorithm that synthesizes sound from silent videos of people hitting and scratching objects with a drumstick. This algorithm uses a recurrent neural network to predict sound features from videos and then produces a waveform from these features with an example-based synthesis procedure. We show that the sounds predicted by our model are realistic enough to fool participants in a "real or fake" psychophysical experiment, and that they convey significant information about material properties and physical interactions

Environmental performance outcomes and indicators for indigenous peoples: Review of literature

The literature review in this report was the starting point for developing a Māori research strand (2003-2009) within the Planning Under Co-operative Mandates (PUCM) research programme (1995-2009). An early task of the PUCM Māori team was to review the international literature on environmental outcomes and indicators for indigenous peoples. This was in order to gain an understanding of what had been written on the subject and to become familiar with approaches taken by others that might provide lessons for the development of our proposed kaupapa Māori outcomes and indicators framework and methodology, which was aimed at local government performance in Aotearoa/New Zealand. This current report is not intended to provide an exhaustive catalogue of writings on environmental performance outcomes and indicators for indigenous peoples, including Māori. Rather, some of the more obvious and important writings are noted as a ready reference for others interested in this topic. Before detailing the approach we took in carrying out the review, the key terms, outcomes and indicators, are defined

Visual to Sound: Generating Natural Sound for Videos in the Wild

As two of the five traditional human senses (sight, hearing, taste, smell, and touch), vision and sound are basic sources through which humans understand the world. Often correlated during natural events, these two modalities combine to jointly affect human perception. In this paper, we pose the task of generating sound given visual input. Such capabilities could help enable applications in virtual reality (generating sound for virtual scenes automatically) or provide additional accessibility to images or videos for people with visual impairments. As a first step in this direction, we apply learning-based methods to generate raw waveform samples given input video frames. We evaluate our models on a dataset of videos containing a variety of sounds (such as ambient sounds and sounds from people/animals). Our experiments show that the generated sounds are fairly realistic and have good temporal synchronization with the visual inputs.Comment: Project page: http://bvision11.cs.unc.edu/bigpen/yipin/visual2sound_webpage/visual2sound.htm