Spatially augmented audio delivery: applications of spatial sound awareness in sensor-equipped indoor environments

Current mainstream audio playback paradigms do not take any account of a user's physical location or orientation in the delivery of audio through headphones or speakers. Thus audio is usually presented as a static perception whereby it is naturally a dynamic 3D phenomenon audio environment. It fails to take advantage of our innate psycho-acoustical perception that we have of sound source locations around us. Described in this paper is an operational platform which we have built to augment the sound from a generic set of wireless headphones. We do this in a way that overcomes the spatial awareness limitation of audio playback in indoor 3D environments which are both location-aware and sensor-equipped. This platform provides access to an audio-spatial presentation modality which by its nature lends itself to numerous cross-dissiplinary applications. In the paper we present the platform and two demonstration applications

An outdoor spatially-aware audio playback platform exemplified by a virtual zoo

Outlined in this short paper is a framework for the construction of outdoor location-and direction-aware audio applications along with an example application to showcase the strengths of the framework and to demonstrate how it works. Although there has been previous work in this area which has concentrated on the spatial presentation of sound through wireless headphones, typically such sounds are presented as though originating from specific, defined spatial locations within a 3D environment. Allowing a user to move freely within this space and adjusting the sound dynamically as we do here, further enhances the perceived reality of the virtual environment. Techniques to realise this are implemented by the real-time adjustment of the presented 2 channels of audio to the headphones, using readings of the user's head orientation and location which in turn are made possible by sensors mounted upon the headphones. Aside from proof of concept indoor applications, more user-responsive applications of spatial audio delivery have not been prototyped or explored. In this paper we present an audio-spatial presentation platform along with a primary demonstration application for an outdoor environment which we call a {\em virtual audio zoo}. This application explores our techniques to further improve the realism of the audio-spatial environments we can create, and to assess what types of future application are possible

Eye fixation related potentials in a target search task

Typically BCI (Brain Computer Interfaces) are found in rehabilitative or restorative applications, often allowing users a medium of communication that is otherwise unavailable through conventional means. Recently, however, there is growing interest in using BCI to assist users in searching for images. A class of neural signals often leveraged in common BCI paradigms are ERPs (Event Related Potentials), which are present in the EEG (Electroencephalograph) signals from users in response to various sensory events. One such ERP is the P300, and is typically elicited in an oddball experiment where a subject’s attention is orientated towards a deviant stimulus among a stream of presented images. It has been shown that these types of neural responses can be used to drive an image search or labeling task, where we can rank images by examining the presence of such ERP signals in response to the display of images. To date, systems like these have been demonstrated when presenting sequences of images containing targets at up to 10Hz, however, the target images in these tasks do not necessitate any kind of eye movement for their detection because the targets in the images are quite salient. In this paper we analyse the presence of discriminating signals when they are offset to the time of eye fixations in a visual search task where detection of target images does require eye fixations

Optimising the number of channels in EEG-augmented image search

Recent proof-of-concept research has appeared showing the applicability of Brain Computer Interface (BCI) technology in combination with the human visual system, to classify images. The basic premise here is that images that arouse a participant’s attention generate a detectable response in their brainwaves, measurable using an electroencephalograph (EEG). When a participant is given a target class of images to search for, each image belonging to that target class presented within a stream of images should elicit a distinctly detectable neural response. Previous work in this domain has primarily focused on validating the technique on proof of concept image sets that demonstrate desired properties and on examining the capabilities of the technique at various image presentation speeds. In this paper we expand on this by examining the capability of the technique when using a reduced number of channels in the EEG, and its impact on the detection accuracy

Spatial Filtering Pipeline Evaluation of Cortically Coupled Computer Vision System for Rapid Serial Visual Presentation

Rapid Serial Visual Presentation (RSVP) is a paradigm that supports the application of cortically coupled computer vision to rapid image search. In RSVP, images are presented to participants in a rapid serial sequence which can evoke Event-related Potentials (ERPs) detectable in their Electroencephalogram (EEG). The contemporary approach to this problem involves supervised spatial filtering techniques which are applied for the purposes of enhancing the discriminative information in the EEG data. In this paper we make two primary contributions to that field: 1) We propose a novel spatial filtering method which we call the Multiple Time Window LDA Beamformer (MTWLB) method; 2) we provide a comprehensive comparison of nine spatial filtering pipelines using three spatial filtering schemes namely, MTWLB, xDAWN, Common Spatial Pattern (CSP) and three linear classification methods Linear Discriminant Analysis (LDA), Bayesian Linear Regression (BLR) and Logistic Regression (LR). Three pipelines without spatial filtering are used as baseline comparison. The Area Under Curve (AUC) is used as an evaluation metric in this paper. The results reveal that MTWLB and xDAWN spatial filtering techniques enhance the classification performance of the pipeline but CSP does not. The results also support the conclusion that LR can be effective for RSVP based BCI if discriminative features are available

Social media marketing in the hospitality industry : is it worth the effort?

This paper explores the impact of hotel social media activity on potential reservations and revenue generation. It does this by firstly exploring the perceptions of senior hotel executives towards the ROI of hotel social media activity. Secondly by data mining hotel reservation data to examine the extent of social media engagement being undertaken by guests with a hotel prior to and post a reservation being made. Thirdly through an experiment using social media advertising to examine its impact on the behaviour of fans and non-fans. The research suggests that social media engagement and advertising do have a positive impact on hotel reservations and revenue generation

Synthetic-Neuroscore: Using A Neuro-AI Interface for Evaluating Generative Adversarial Networks

Generative adversarial networks (GANs) are increasingly attracting attention in the computer vision, natural language processing, speech synthesis and similar domains. Arguably the most striking results have been in the area of image synthesis. However, evaluating the performance of GANs is still an open and challenging problem. Existing evaluation metrics primarily measure the dissimilarity between real and generated images using automated statistical methods. They often require large sample sizes for evaluation and do not directly reflect human perception of image quality. In this work, we describe an evaluation metric we call Neuroscore, for evaluating the performance of GANs, that more directly reflects psychoperceptual image quality through the utilization of brain signals. Our results show that Neuroscore has superior performance to the current evaluation metrics in that: (1) It is more consistent with human judgment; (2) The evaluation process needs much smaller numbers of samples; and (3) It is able to rank the quality of images on a per GAN basis. A convolutional neural network (CNN) based neuro-AI interface is proposed to predict Neuroscore from GAN-generated images directly without the need for neural responses. Importantly, we show that including neural responses during the training phase of the network can significantly improve the prediction capability of the proposed model. Materials related to this work are provided at https://github.com/villawang/Neuro-AI-Interface

Neurological modeling of what experts vs. non-experts find interesting

The P3 and related ERP's have a long history of use to identify stimulus events in subjects as part of oddball-style experiments. In this work we describe the ongoing development of oddball style experiments which attempt to capture what a subject finds of interest or curious, when presented with a set of visual stimuli i.e. images. This joint work between Dublin City University (DCU) and the European Space Agency's Advanced Concepts Team (ESA ACT) is motivated by the challenges of autonomous space exploration where the time lag for sending data back to earth for analysis and then communicating an action or decision back to the spacecraft means that decision-making is slow. Also, when extraterrestrial sensors capture data, the determination of what data to send back to earth is driven by an expertly devised rule set, that is scientists need to determine apriori what will be of interest. This cannot adapt to novel or unexpected data that a scientist may find curious. Our work is attempting to determine if it is possible to capture what a scientist (subject) finds of interest (curious) in a stream of image data through EEG measurement. One of the our challenges is to determine the difference between an expert and a lay subject response to stimulus. To investigate the theorized difference, we use a set of lifelog images as our dataset. Lifelog images are first person images taken by a small wearable camera which continuously records images whilst it is worn. We have devised two key experiments for use with this data and two classes of subjects. Our subjects are a person who has worn the personal camera, from which our collection of lifelog images is taken and who becomes our expert, and the remaining subjects are people who have no association with the captured images. Our first experiment is a traditional oddball experiment where the oddballs are people having coffee, and can be thought of as a directed information seeking task. The second experiment is to present a stream of lifelog images to the subjects and record which images cause a stimulus response. Once the data from these experiments has been captured our task is to compare the responses between the expert and lay subject groups, to determine if there are any commonalities between these groups or any distinct differences. If the latter outcome is the case the objective is then to investigate methods for capturing properties of images which cause an expert to be interested in a presented image. Further novelty is added to our work by the fact we are using entry-level off-the-shelf EEG devices, consisting of 4 nodes with a sampling rate of 255Hz

Stereochemical assignment of the protein-protein interaction inhibitor JBIR-22 by total synthesis

The authors acknowledge the EPSRC and Cancer Research UK (CRUK Grant No. C21383/A6950) for funding this research.Recent reports have highlighted the biological activity associated with a sub-family of the tetramic acid class of natural products. Despite the fact that members of this sub-family act as protein-protein interaction inhibitors of relevance to proteasome assembly, no synthetic work has been reported. This may be because this sub-family contains an unnatural 4,4-disubstitued glutamic acid, the synthesis of which provides a key challenge. Here we describe a highly stereoselective route to a masked form of this unnatural amino acid. This enabled the synthesis of two of the possible diastereomers of JBIR-22 and allowed its relative and absolute stereochemistry to be assigned.Publisher PDFPeer reviewe

Lifelogging and EEG: utilising neural signals for sorting lifelog image data

Lifelogging– particularly image capture – is capable of generating vast amounts of image data of complex human activates and events which can be difficult to automatically sort and navigate. In this work we demonstrate how neural signals from EEG (Electroencephalography) can be used to help sort and navigate these datasets at high speed. By using EEG we can detect a variety of attention related neural responses to viewing lifelogimages which in turn allows us to sort them from the subjective perspective of which images caught the person’s attention most significantly