Detecting and indexing moving objects for Behavior Analysis by Video and Audio Interpretation

2012 - 2013In the last decades we have assisted to a growing need for security in many public environments. According to a study recently conducted by the European Security Observatory, one half of the entire population is worried about the crime and requires the law enforcement to be protected. This consideration has lead the proliferation of cameras and microphones, which represent a suitable solution for their relative low cost of maintenance, the possibility of installing them virtually everywhere and, finally, the capability of analysing more complex events. However, the main limitation of this traditional audiovideo surveillance systems lies in the so called psychological overcharge issue of the human operators responsible for security, that causes a decrease in their capabilities to analyse raw data flows from multiple sources of multimedia information; indeed, as stated by a study conducted by Security Solutions magazine, after 12 minutes of continuous video monitoring, a guard will often miss up to 45% of screen activity. After 22 minutes of video, up to 95% is overlooked. For the above mentioned reasons, it would be really useful to have available an intelligent surveillance system, able to provide images and video with a semantic interpretation, for trying to bridge the gap between their low-level representation in terms of pixels, and the high-level, natural language description that a human would give about them. On the other hand, this kind of systems, able to automatically understand the events occurring in a scene, would be really useful in other application fields, mainly oriented to marketing purposes. Especially in the last years, a lot of business intelligent applications have been installed for assisting decision makers and for giving an organization’s employees, partners and suppliers easy access to the information they need to effectively do their jobs... [edited by author]XII n.s

Video Abstracting at a Semantical Level

One the most common form of a video abstract is the movie trailer. Contemporary movie trailers share a common structure across genres which allows for an automatic generation and also reflects the corresponding moviea s composition. In this thesis a system for the automatic generation of trailers is presented. In addition to action trailers, the system is able to deal with further genres such as Horror and comedy trailers, which were first manually analyzed in order to identify their basic structures. To simplify the modeling of trailers and the abstract generation itself a new video abstracting application was developed. This application is capable of performing all steps of the abstract generation automatically and allows for previews and manual optimizations. Based on this system, new abstracting models for horror and comedy trailers were created and the corresponding trailers have been automatically generated using the new abstracting models. In an evaluation the automatic trailers were compared to the original Trailers and showed a similar structure. However, the automatically generated trailers still do not exhibit the full perfection of the Hollywood originals as they lack intentional storylines across shots

Discrimination of age, sex, and individual identity using the upcall of the North Atlantic right whale (Eubalaena glacialis)

According to the source-filter theory proposed for human speech, physical attributes of the mammalian vocal production mechanism combine independently to result in individually distinctive vocalizations. In the case of stereotyped calls with all individuals producing a similar frequency contour, formants resulting from the shape and size of the vocal tract may be more likely to contain individually distinctive information than the fundamental frequency resulting from the vibrating source. However, the formant structure resulting from such filtering has been historically undervalued in the majority of studies addressing individual distinctiveness in non-human species. The upcall of the North Atlantic right whale (Eubalaena glacialis) is characterized as a stereotyped contact call, and visual inspection of upcall spectrograms confirms presence of a robust formant structure. Here I present results testing age, sex, and individual distinctiveness of upcalls recorded from archival, suction cup mounted tags (Dtags). Multiple measurements were made using the fundamental frequency contour, formant structure, and amplitude of the upcalls. These three variable groupings were then tested alone and in combination with other groupings to assign upcalls to age classes based on reproductive maturity, age classes based on size, sex, and individual whales. To compare multiple classification methods, I used both discriminant function analysis and a classification and regression tree to classify calls to appropriate groups. In all analyses, the percentage of calls correctly assigned to the correct group—age, sex, individual—was significantly higher than chance levels. These results represent the first quantitative analysis of individual distinctiveness in mysticete whales and provide a baseline for further development of acoustic detection techniques that could be used to noninvasively track movements of whales across multiple habitats

Machine therapy

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2007.Includes bibliographical references (p. [137]-146).Machine Therapy is a new practice combining art, design, psychoanalysis, and engineering work in ways that access and reveal the vital, though often unnoticed, relevance of people's interactions and relationships with machines. Machine Therapy will be illustrated through the construction of several systems including re-appropriated domestic devices such as Blendie, wearble apparatuses such as ScreamBody, and body-signal-based companion machines - Umo, Amo, and Omo - that function through visceral interactions including breathing and non-verbal sounds. These systems will be used to explore themes of human-machine relations in terms of visceral, cathartic, and reflexive expressions and communications. This work incorporates elements from my technical research in digital signal processing, machine learning, mechanical engineering, and sensor design. Combining these areas of research and practice, I have been able to help manifest new objects and relationships that are unique in some aspects while maintaining quotidian familiarity in other aspects. These apparatuses enable unusual explorations of what we interact with when we interact with machines. I hypothesize that the answer will turn out to be much more than the machine itself, and will include our sense of self, agency in the interpersonal and political world, and our shared psychological, emotional, cultural, and perceptual approaches to the world. The importance of the parapractic elements and also the therapeutic properties of the Machine Therapy machines will be evaluated in studies of participants' interactive engagements with the machines as well as their affective responses to the machines.Kelly Dobson.Ph.D