Analysis and resynthesis of polyphonic music

This thesis examines applications of Digital Signal Processing to the analysis, transformation, and resynthesis of musical audio. First I give an overview of the human perception of music. I then examine in detail the requirements for a system that can analyse, transcribe, process, and resynthesise monaural polyphonic music. I then describe and compare the possible hardware and software platforms. After this I describe a prototype hybrid system that attempts to carry out these tasks using a method based on additive synthesis. Next I present results from its application to a variety of musical examples, and critically assess its performance and limitations. I then address these issues in the design of a second system based on Gabor wavelets. I conclude by summarising the research and outlining suggestions for future developments

Video big data: an agile architecture for systematic exploration and analytics

Video is currently at the forefront of most business and natural environments. In surveillance, it is the most important technology as surveillance systems reveal information and patterns for solving many security problems including crime prevention. This research investigates technologies that currently drive video surveillance systems with a view to optimization and automated decision support. The investigation reveals some features and properties that can be optimised to improve performance and derive further benefits from surveillance systems. These aspects include system-wide architecture, meta-data generation, meta-data persistence, object identification, object tagging, object tracking, search and querying sub-systems. The current less-than-optimum performance is attributable to many factors, which include massive volume, variety, and velocity (the speed at which streaming video transmit to storage) of video data in surveillance systems. Research contributions are 2-fold. First, we propose a system-wide architecture for designing and implementing surveillance systems, based on the authors’ system architecture for generating meta-data. Secondly, we design a simulation model of a multi-view surveillance system from which the researchers generate simulated video streams in large volumes. From each video sequence in the model, the authors extract meta-data and apply a novel algorithm for predicting the location of identifiable objects across a well-connected camera cluster. This research provide evidence that independent surveillance systems (for example, security cameras) can be unified across a geographical location such as a smart city, where each network is administratively owned and managed independently. Our investigation involved 2 experiments - first, the implementation of a web-based solution where we developed a directory service for managing, cataloguing, and persisting metadata generated by the surveillance networks. The second experiment focused on the set up, configuration and the architecture of the surveillance system. These experiments involved the investigation and demonstration of 3 loosely coupled service-oriented APIs – these services provided the capability to generate the query-able metadata. The results of our investigations provided answers to our research questions - the main question being “to what degree of accuracy can we predict the location of an object in a connected surveillance network”. Our experiment also provided evidence in support of our hypothesis – “it is feasible to ‘explore’ unified surveillance data generated from independent surveillance networks”

Design and Implementation of a Middleware for Uniform, Federated and Dynamic Event Processing

In recent years, real-time processing of massive event streams has become an important topic in the area of data analytics. It will become even more important in the future due to cheap sensors, a growing amount of devices and their ubiquitous inter-connection also known as the Internet of Things (IoT). Academia, industry and the open source community have developed several event processing (EP) systems that allow users to define, manage and execute continuous queries over event streams. They achieve a significantly better performance than the traditional store-then-process'' approach in which events are first stored and indexed in a database. Because EP systems have different roots and because of the lack of standardization, the system landscape became highly heterogenous. Today's EP systems differ in APIs, execution behaviors and query languages. This thesis presents the design and implementation of a novel middleware that abstracts from different EP systems and provides a uniform API, execution behavior and query language to users and developers. As a consequence, the presented middleware overcomes the problem of vendor lock-in and different EP systems are enabled to cooperate with each other. In practice, event streams differ dramatically in volume and velocity. We show therefore how the middleware can connect to not only different EP systems, but also database systems and a native implementation. Emerging applications such as the IoT raise novel challenges and require EP to be more dynamic. We present extensions to the middleware that enable self-adaptivity which is needed in context-sensitive applications and those that deal with constantly varying sets of event producers and consumers. Lastly, we extend the middleware to fully support the processing of events containing spatial data and to be able to run distributed in the form of a federation of heterogenous EP systems

Personal Wayfinding Assistance

We are traveling many different routes every day. In familiar environments it is easy for us to find our ways. We know our way from bedroom to kitchen, from home to work, from parking place to office, and back home at the end of the working day. We have learned these routes in the past and are now able to find our destination without having to think about it. As soon as we want to find a place beyond the demarcations of our mental map, we need help. In some cases we ask our friends to explain us the way, in other cases we use a map to find out about the place. Mobile phones are increasingly equipped with wayfinding assistance. These devices are usually at hand because they are handy and small, which enables us to get wayfinding assistance everywhere where we need it. While the small size of mobile phones makes them handy, it is a disadvantage for displaying maps. Geographic information requires space to be visualized in order to be understandable. Typically, not all information displayed in maps is necessary. An example are walking ways in parks for car drivers, they are they are usually no relevant route options. By not displaying irrelevant information, it is possible to compress the map without losing important information. To reduce information purposefully, we need information about the user, the task at hand, and the environment it is embedded in. In this cumulative dissertation, I describe an approach that utilizes the prior knowledge of the user to adapt maps to the to the limited display options of mobile devices with small displays. I focus on central questions that occur during wayfinding and relate them to the knowledge of the user. This enables the generation of personal and context-specific wayfinding assistance in the form of maps which are optimized for small displays. To achieve personalized assistance, I present algorithmic methods to derive spatial user profiles from trajectory data. The individual profiles contain information about the places users regularly visit, as well as the traveled routes between them. By means of these profiles it is possible to generate personalized maps for partially familiar environments. Only the unfamiliar parts of the environment are presented in detail, the familiar parts are highly simplified. This bears great potential to minimize the maps, while at the same time preserving the understandability by including personally meaningful places as references. To ensure the understandability of personalized maps, we have to make sure that the names of the places are adapted to users. In this thesis, we study the naming of places and analyze the potential to automatically select and generate place names. However, personalized maps only work for environments the users are partially familiar with. If users need assistance for unfamiliar environments, they require complete information. In this thesis, I further present approaches to support uses in typical situations which can occur during wayfinding. I present solutions to communicate context information and survey knowledge along the route, as well as methods to support self-localization in case orientation is lost

Ubiquitous User Modeling

More and more interactions take place between humans and mobile or connected IT-systems in daily life. This offers a great opportunity, especially to user modeling, to reach better adaptation with ongoing evaluation of user behavior. This work develops a complete framework to realize the newly defined concept of ubiquitous user modeling. The developed tools cover methods for the uniform exchange and the semantic integration of partial user models. They also account for the extended needs for privacy and the right of every human for introspection and control of their collected data. The SITUATIONALSTATEMENTS and the exchange language USERML have been developed on the syntactical level, while the general user model ontology GUMO and the UBISWORLD ontology have been developed on the semantical level. A multilevel conflict resolution method, which handles the problem of contradictory statements, has been implemented together with a web-based user model service, such that the road capability and the scalability can be proven with this approach.Immer häufiger auftretende Interaktionen im täglichen Leben zwischen Menschen und vernetzten oder mobilen IT-Systemen bieten insbesondere für die Benutzermodellierung eine große Chance, durch ständige Evaluation des Benutzerverhaltens verbesserte Adaptionsleistungen zu erzielen. Die vorliegende Arbeit entwickelt ein komplettes Rahmensystem, um dieses neu definierte Konzept der ubiquitären Benutzermodellierung zu realisieren. Die erarbeiteten Werkzeuge umfassen Methoden zum einheitlichen Austausch und zur semantischen Integration von partiellen Benutzermodellen. Sie berücksichtigen aber auch die erhöhten Anforderungen an die Privatsphäre, sowie das Recht der Menschen auf Introspektion und Kontrolle über die erhobenen Daten. Auf syntaktischer Ebene werden die situationsbeschreibenden Aussagen sowie die Austauschsprache UserML entworfen. Auf semantischer Ebene werden die allgemeine Benutzermodell-Ontologie GUMO und die UBISWELT-Ontologie entwickelt. Ein mehrstufiger Konfliktlösungsmechanismus, der das Problem sich widersprechender Aussagen bearbeitet, wird zusammen mit einem webbasierten Benutzermodell-Service implementiert, sodass die Praxistauglichkeit und die Skalierbarkeit dieses Ansatzes an mehreren Beispielen gezeigt werden kann

Smartphones: A Platform For Disaster Management

Bal, H.E. [Promotor]Kielmann, T. [Copromotor