Crowd-Sourcing the Sounds of Places with a Web-Based Evolutionary Algorithm

The sounds we associate with particular places are tightly interwoven with our memories and sense of belonging. We describe a platform designed to assist in gathering the sounds a group of people associate with a place. A web-based evolutionary algorithm, with human-in-the-loop fitness evaluations, ranks and recombines sounds to find collections that the group rates as familiar. An experiment covering four geographical locations shows that the process does indeed find sounds deemed familiar by participants

Crowd Sourcing The Sounds Of Places With A Web Based Genetic Algorithm Techreport

The sounds that we associate with particular places are tightly interwoven with our memories and sense of belonging. It is assumed that such an association is a complex process, and much of its mechanism is hidden from analytical examination. The association of sound to place has been much explored and examined by artistic approaches. For example, soundscape composition, which makes great use of recorded and barely-processed sounds from places in the compositional practice, highlights the power of the association. However, it does not offer us a scientific insight into its process, particularly, the role of familiarity of sounds people hear and their association to specific places. We describe a platform designed to assist in gathering the sounds that a group of people associate with a place. A web-based evolutionary algorithm, with human-in-the-loop fitness evaluations, ranks and recombines sounds to find collections that the group rates as familiar. An experiment involving independent groups of people associated with four geographical locations shows that the process does indeed find sounds deemed familiar by participants

Worldwide Infrastructure for Neuroevolution: A Modular Library to Turn Any Evolutionary Domain into an Online Interactive Platform

Across many scientific disciplines, there has emerged an open opportunity to utilize the scale and reach of the Internet to collect scientific contributions from scientists and non-scientists alike. This process, called citizen science, has already shown great promise in the fields of biology and astronomy. Within the fields of artificial life (ALife) and evolutionary computation (EC) experiments in collaborative interactive evolution (CIE) have demonstrated the ability to collect thousands of experimental contributions from hundreds of users across the glob. However, such collaborative evolutionary systems can take nearly a year to build with a small team of researchers. This dissertation introduces a new developer framework enabling researchers to easily build fully persistent online collaborative experiments around almost any evolutionary domain, thereby reducing the time to create such systems to weeks for a single researcher. To add collaborative functionality to any potential domain, this framework, called Worldwide Infrastructure for Neuroevolution (WIN), exploits an important unifying principle among all evolutionary algorithms: regardless of the overall methods and parameters of the evolutionary experiment, every individual created has an explicit parent-child relationship, wherein one individual is considered the direct descendant of another. This principle alone is enough to capture and preserve the relationships and results for a wide variety of evolutionary experiments, while allowing multiple human users to meaningfully contribute. The WIN framework is first validated through two experimental domains, image evolution and a new two-dimensional virtual creature domain, Indirectly Encoded SodaRace (IESoR), that is shown to produce a visually diverse variety of ambulatory creatures. Finally, an Android application built with WIN, filters, allows users to interactively evolve custom image effects to apply to personalized photographs, thereby introducing the first CIE application available for any mobile device. Together, these collaborative experiments and new mobile application establish a comprehensive new platform for evolutionary computation that can change how researchers design and conduct citizen science online

Improving the accuracy and efficiency of bird song analysis with machine learning based event detection

Ornithology is an exciting field with novel research emerging everyday. Researchers in bioacoustics often spend hours within the wilderness recording bird calls to analyze later in their lab. The burden of sifting through hours of audio recordings from the field continues to remain a time-consuming and manual task, despite the inherent value of such research. In this work, we investigate the ability of a mobile application to provide an accelerated method for ornithologists to upload acoustic field recordings for analysis. Driven by recent advancements in technology, there is a growing need for innovative solutions that can streamline this research process. Our proposed solution integrates machine learning algorithms with bioacoustic signal processing with the goal of accelerating the process of analyzing hours of field recordings. In this thesis, we focus on the integration of future machine learning algorithms into an event detection pipeline. This larger project involves three separate parts: (1) the user interface and back end architecture developed by a small team of students for the Oregon State CS 46x capstone series, (2) an event detection algorithm completed as a synergistic extension of the capstone, and (3) a machine learning system developed as part of a graduate thesis. Together, these projects integrate to enhance the accuracy and efficiency of bioacoustic event detection by leveraging machine intelligence with deep learning. Furthermore, we evaluate the performance of these approaches using real-world datasets and discuss our approach in the context of traditional event detection techniques. This research project contributes valuable insights into the feasibility and effectiveness of integrating machine learning algorithms into a cloud-based acoustic event detection pipeline and highlights the potential benefits of this integration for various applications to expedite time-consuming bioacoustic research processes.Key Words: audio processing, bioacoustics, bird song, event detection, databas

Composing affect: reflection on configurations of body, sound and technology in contemporary South African performance

This thesis engages with experiential performance modes through the lenses of phenomenology and affect theory. Because experiential performance relies per definition on personal, subjective ‘experience’, specific responses cannot be anticipated. However, by attempting to compose ‘affect’, a performance has the potential to ‘move’ an attendant towards response. Deleuze and Guattari define ‘affect’ as “an ability to affect and be affected….a prepersonal intensity corresponding to the passage from one experiential state of the body to another and implying an augmentation or diminution in that body’s capacity to act” (1987: xvi). One current strategy for manifesting affect in performance seems to be the ways in which different configurations of body, sound and technology are employed. The body is the means through which sound is received or ‘experienced’ in the phenomenological sense, but it can also act as a source for sonic material. The body is furthermore the means by which sonic technology is manipulated. It is the complex, reverberating relationships between body, sound and technology, and their potential for eliciting affective transformation, which is the focus of my enquiry. In the first chapter I unpack the roles of the natural phenomena, body and sound, and their complex relationships to affect. The chapter serves as philosophical basis for the rest of the investigation, and draws largely on works by philosophers Susan Kozel, Maurice Merleau-Ponty, Brian Massumi, Gille Deleuze and Félix Guatarri and sound theorists Don Ihde, Marshall McLuhan, Brandon LaBelle and Frances Dyson.In the remaining three chapters I discuss current South African theatre works that employ the strategy of placing emphasis on sound, sonic technology, and its relationship to the human body. These works are my own piece herTz (2014), Jaco Bouwer’s pieces Samsa-masjien (2014) and Na-aap (2013), and First Physical Theatre Company’s Everyday Falling (2010). While they range from being plays to physical theatre performances to performative experiments, they all place specific emphasis on sonic devices, drawing attention to sound by revealing microphones, speakers, midi boards, etc. to the attendants, and including the generation and manipulation of sound in the action of the performance

Mobile phone technology as an aid to contemporary transport questions in walkability, in the context of developing countries

The emerging global middle class, which is expected to double by 2050 desires more walkable, liveable neighbourhoods, and as distances between work and other amenities increases, cities are becoming less monocentric and becoming more polycentric. African cities could be described as walking cities, based on the number of people that walk to their destinations as opposed to other means of mobility but are often not walkable. Walking is by far the most popular form of transportation in Africa’s rapidly urbanising cities, although it is not often by choice rather a necessity. Facilitating this primary mode, while curbing the growth of less sustainable mobility uses requires special attention for the safety and convenience of walking in view of a Global South context. In this regard, to further promote walking as a sustainable mobility option, there is a need to assess the current state of its supporting infrastructure and begin giving it higher priority, focus and emphasis. Mobile phones have emerged as a useful alternative tool to collect this data and audit the state of walkability in cities. They eliminate the inaccuracies and inefficiencies of human memories because smartphone sensors such as GPS provides information with accuracies within 5m, providing superior accuracy and precision compared to other traditional methods. The data is also spatial in nature, allowing for a range of possible applications and use cases. Traditional inventory approaches in walkability often only revealed the perceived walkability and accessibility for only a subset of journeys. Crowdsourcing the perceived walkability and accessibility of points of interest in African cities could address this, albeit aspects such as ease-of-use and road safety should also be considered. A tool that crowdsources individual pedestrian experiences; availability and state of pedestrian infrastructure and amenities, using state-of-the-art smartphone technology, would over time also result in complete surveys of the walking environment provided such a tool is popular and safe. This research will illustrate how mobile phone applications currently in the market can be improved to offer more functionality that factors in multiple sensory modalities for enhanced visual appeal, ease of use, and aesthetics. The overarching aim of this research is, therefore, to develop the framework for and test a pilot-version mobile phone-based data collection tool that incorporates emerging technologies in collecting data on walkability. This research project will assess the effectiveness of the mobile application and test the technical capabilities of the system to experience how it operates within an existing infrastructure. It will continue to investigate the use of mobile phone technology in the collection of user perceptions of walkability, and the limitations of current transportation-based mobile applications, with the aim of developing an application that is an improvement to current offerings in the market. The prototype application will be tested and later piloted in different locations around the globe. Past studies are primarily focused on the development of transport-based mobile phone applications with basic features and limited functionality. Although limited progress has been made in integrating emerging advanced technologies such as Augmented Reality (AR), Machine Learning (ML), Big Data analytics, amongst others into mobile phone applications; what is missing from these past examples is a comprehensive and structured application in the transportation sphere. In turn, the full research will offer a broader understanding of the iii information gathered from these smart devices, and how that large volume of varied data can be better and more quickly interpreted to discover trends, patterns, and aid in decision making and planning. This research project attempts to fill this gap and also bring new insights, thus promote the research field of transportation data collection audits, with particular emphasis on walkability audits. In this regard, this research seeks to provide insights into how such a tool could be applied in assessing and promoting walkability as a sustainable and equitable mobility option. In order to get policy-makers, analysts, and practitioners in urban transport planning and provision in cities to pay closer attention to making better, more walkable places, appealing to them from an efficiency and business perspective is vital. This crowdsourced data is of great interest to industry practitioners, local governments and research communities as Big Data, and to urban communities and civil society as an input in their advocacy activities. The general findings from the results of this research show clear evidence that transport-based mobile phone applications currently available in the market are increasingly getting outdated and are not keeping up with new and emerging technologies and innovations. It is also evident from the results that mobile smartphones have revolutionised the collection of transport-related information hence the need for new initiatives to help take advantage of this emerging opportunity. The implications of these findings are that more attention needs to be paid to this niche going forward. This research project recommends that more studies, particularly on what technologies and functionalities can realistically be incorporated into mobile phone applications in the near future be done as well as on improving the hardware specifications of mobile phone devices to facilitate and support these emerging technologies whilst keeping the cost of mobile devices as low as possible

Parameter Search for Aesthetic Design and Composition

PhDThis thesis is about algorithmic creation in the arts – where an artist, designer or composer uses a formal generative process to assist in crafting forms and patterns – and approaches to finding effective input parameter values to these generative processes for aesthetic ends. Framed in three practical studies, approaches to navigating the aesthetic possibilities of generative processes in sound and visuals are presented, and strategies for eliciting the preferences of the consumers of the generated output are explored. The first study presents a musical interface that enables navigation of the possibilities of a stochastic generative process with respect to measures of subjective predictability. Through a mobile phone version of the application, aesthetic preferences are crowd-sourced. The second study presents an eye-tracking based framework for the exploration of the possibilities afforded by generative designs; the interaction between the viewers’ gaze patterns and the system engendering a fluid navigation of the state-space of the visual forms. The third study presents a crowd-sourced interactive evolutionary system, where populations of abstract colour images are shaped by thousands of preference selections from users worldwide For each study, the results of analyses eliciting the attributes of the generated outputs – and their associated parameter values – that are most preferred by the consumers/users of these systems are presented. Placed in a historical and theoretical context, a refined perspective on the complex interrelationships between generative processes, input parameters and perceived aesthetic value is presented. Contributions to knowledge include identified trends in objective aesthetic preferences in colour combinations and their arrangements, theoretical insights relating perceptual mechanisms to generative system design and analysis, strategies for effectively leveraging evolutionary computation in an empirical aesthetic context, and a novel eye-tracking based framework for the exploration of visual generative designs.Engineering and Physical Sciences Research Council (EPSRC) as part of the Doctoral Training Centre in Media and Arts Technology at Queen Mary University of London (ref: EP/G03723X/1)

Application of Common Sense Computing for the Development of a Novel Knowledge-Based Opinion Mining Engine

The ways people express their opinions and sentiments have radically changed in the past few years thanks to the advent of social networks, web communities, blogs, wikis and other online collaborative media. The distillation of knowledge from this huge amount of unstructured information can be a key factor for marketers who want to create an image or identity in the minds of their customers for their product, brand, or organisation. These online social data, however, remain hardly accessible to computers, as they are specifically meant for human consumption. The automatic analysis of online opinions, in fact, involves a deep understanding of natural language text by machines, from which we are still very far. Hitherto, online information retrieval has been mainly based on algorithms relying on the textual representation of web-pages. Such algorithms are very good at retrieving texts, splitting them into parts, checking the spelling and counting their words. But when it comes to interpreting sentences and extracting meaningful information, their capabilities are known to be very limited. Existing approaches to opinion mining and sentiment analysis, in particular, can be grouped into three main categories: keyword spotting, in which text is classified into categories based on the presence of fairly unambiguous affect words; lexical affinity, which assigns arbitrary words a probabilistic affinity for a particular emotion; statistical methods, which calculate the valence of affective keywords and word co-occurrence frequencies on the base of a large training corpus. Early works aimed to classify entire documents as containing overall positive or negative polarity, or rating scores of reviews. Such systems were mainly based on supervised approaches relying on manually labelled samples, such as movie or product reviews where the opinionist’s overall positive or negative attitude was explicitly indicated. However, opinions and sentiments do not occur only at document level, nor they are limited to a single valence or target. Contrary or complementary attitudes toward the same topic or multiple topics can be present across the span of a document. In more recent works, text analysis granularity has been taken down to segment and sentence level, e.g., by using presence of opinion-bearing lexical items (single words or n-grams) to detect subjective sentences, or by exploiting association rule mining for a feature-based analysis of product reviews. These approaches, however, are still far from being able to infer the cognitive and affective information associated with natural language as they mainly rely on knowledge bases that are still too limited to efficiently process text at sentence level. In this thesis, common sense computing techniques are further developed and applied to bridge the semantic gap between word-level natural language data and the concept-level opinions conveyed by these. In particular, the ensemble application of graph mining and multi-dimensionality reduction techniques on two common sense knowledge bases was exploited to develop a novel intelligent engine for open-domain opinion mining and sentiment analysis. The proposed approach, termed sentic computing, performs a clause-level semantic analysis of text, which allows the inference of both the conceptual and emotional information associated with natural language opinions and, hence, a more efficient passage from (unstructured) textual information to (structured) machine-processable data. The engine was tested on three different resources, namely a Twitter hashtag repository, a LiveJournal database and a PatientOpinion dataset, and its performance compared both with results obtained using standard sentiment analysis techniques and using different state-of-the-art knowledge bases such as Princeton’s WordNet, MIT’s ConceptNet and Microsoft’s Probase. Differently from most currently available opinion mining services, the developed engine does not base its analysis on a limited set of affect words and their co-occurrence frequencies, but rather on common sense concepts and the cognitive and affective valence conveyed by these. This allows the engine to be domain-independent and, hence, to be embedded in any opinion mining system for the development of intelligent applications in multiple fields such as Social Web, HCI and e-health. Looking ahead, the combined novel use of different knowledge bases and of common sense reasoning techniques for opinion mining proposed in this work, will, eventually, pave the way for development of more bio-inspired approaches to the design of natural language processing systems capable of handling knowledge, retrieving it when necessary, making analogies and learning from experience

Application of Common Sense Computing for the Development of a Novel Knowledge-Based Opinion Mining Engine

The ways people express their opinions and sentiments have radically changed in the past few years thanks to the advent of social networks, web communities, blogs, wikis and other online collaborative media. The distillation of knowledge from this huge amount of unstructured information can be a key factor for marketers who want to create an image or identity in the minds of their customers for their product, brand, or organisation. These online social data, however, remain hardly accessible to computers, as they are specifically meant for human consumption. The automatic analysis of online opinions, in fact, involves a deep understanding of natural language text by machines, from which we are still very far. Hitherto, online information retrieval has been mainly based on algorithms relying on the textual representation of web-pages. Such algorithms are very good at retrieving texts, splitting them into parts, checking the spelling and counting their words. But when it comes to interpreting sentences and extracting meaningful information, their capabilities are known to be very limited. Existing approaches to opinion mining and sentiment analysis, in particular, can be grouped into three main categories: keyword spotting, in which text is classified into categories based on the presence of fairly unambiguous affect words; lexical affinity, which assigns arbitrary words a probabilistic affinity for a particular emotion; statistical methods, which calculate the valence of affective keywords and word co-occurrence frequencies on the base of a large training corpus. Early works aimed to classify entire documents as containing overall positive or negative polarity, or rating scores of reviews. Such systems were mainly based on supervised approaches relying on manually labelled samples, such as movie or product reviews where the opinionist’s overall positive or negative attitude was explicitly indicated. However, opinions and sentiments do not occur only at document level, nor they are limited to a single valence or target. Contrary or complementary attitudes toward the same topic or multiple topics can be present across the span of a document. In more recent works, text analysis granularity has been taken down to segment and sentence level, e.g., by using presence of opinion-bearing lexical items (single words or n-grams) to detect subjective sentences, or by exploiting association rule mining for a feature-based analysis of product reviews. These approaches, however, are still far from being able to infer the cognitive and affective information associated with natural language as they mainly rely on knowledge bases that are still too limited to efficiently process text at sentence level. In this thesis, common sense computing techniques are further developed and applied to bridge the semantic gap between word-level natural language data and the concept-level opinions conveyed by these. In particular, the ensemble application of graph mining and multi-dimensionality reduction techniques on two common sense knowledge bases was exploited to develop a novel intelligent engine for open-domain opinion mining and sentiment analysis. The proposed approach, termed sentic computing, performs a clause-level semantic analysis of text, which allows the inference of both the conceptual and emotional information associated with natural language opinions and, hence, a more efficient passage from (unstructured) textual information to (structured) machine-processable data. The engine was tested on three different resources, namely a Twitter hashtag repository, a LiveJournal database and a PatientOpinion dataset, and its performance compared both with results obtained using standard sentiment analysis techniques and using different state-of-the-art knowledge bases such as Princeton’s WordNet, MIT’s ConceptNet and Microsoft’s Probase. Differently from most currently available opinion mining services, the developed engine does not base its analysis on a limited set of affect words and their co-occurrence frequencies, but rather on common sense concepts and the cognitive and affective valence conveyed by these. This allows the engine to be domain-independent and, hence, to be embedded in any opinion mining system for the development of intelligent applications in multiple fields such as Social Web, HCI and e-health. Looking ahead, the combined novel use of different knowledge bases and of common sense reasoning techniques for opinion mining proposed in this work, will, eventually, pave the way for development of more bio-inspired approaches to the design of natural language processing systems capable of handling knowledge, retrieving it when necessary, making analogies and learning from experience