Implicit Retrieval Of Salient Images Using Brain Computer Interface

Space missions are often equipped with several high definition sensors that can autonomously collect a potentially enormous amount of data. The bottleneck in retrieving these often precious datasets is the onboard data storing capability and the communication bandwidth, which limit the amount of data that can be sent back to Earth. In this paper, we propose a method based on the analysis of brain electrical activity to identify the scientific interest of experts towards a given image in a large set of images. Such a method can be used to efficiently create an abundant training set (images and whether they are scientifically interesting) with a considerably faster image presentation rate that can go beyond expert consciousness, with less interrogation time for experts and relatively high performance

Implicit retrieval of salient images using brain computer interface

ABSTRACT Space missions are often equipped with several high definition sensors that can autonomously collect a potentially enormous amount of data. The bottleneck in retrieving these often precious datasets is the onboard data storing capability and the communication bandwidth, which limit the amount of data that can be sent back to Earth. In this paper, we propose a method based on the analysis of brain electrical activity to identify the scientific interest of experts towards a given image in a large set of images. Such a method can be used to efficiently create an abundant training set (images and whether they are scientifically interesting) with a considerably faster image presentation rate that can go beyond expert consciousness, with less interrogation time for experts and relatively high performance

Object Segmentation in Images using EEG Signals

This paper explores the potential of brain-computer interfaces in segmenting objects from images. Our approach is centered around designing an effective method for displaying the image parts to the users such that they generate measurable brain reactions. When an image region, specifically a block of pixels, is displayed we estimate the probability of the block containing the object of interest using a score based on EEG activity. After several such blocks are displayed, the resulting probability map is binarized and combined with the GrabCut algorithm to segment the image into object and background regions. This study shows that BCI and simple EEG analysis are useful in locating object boundaries in images.Comment: This is a preprint version prior to submission for peer-review of the paper accepted to the 22nd ACM International Conference on Multimedia (November 3-7, 2014, Orlando, Florida, USA) for the High Risk High Reward session. 10 page

Improving object segmentation by using EEG signals and rapid serial visual presentation

This paper extends our previous work on the potential of EEG-based brain computer interfaces to segment salient objects in images. The proposed system analyzes the Event Related Potentials (ERP) generated by the rapid serial visual presentation of windows on the image. The detection of the P300 signal allows estimating a saliency map of the image, which is used to seed a semi-supervised object segmentation algorithm. Thanks to the new contributions presented in this work, the average Jaccard index was improved from $0.47$ to $0.66$ when processed in our publicly available dataset of images, object masks and captured EEG signals. This work also studies alternative architectures to the original one, the impact of object occupation in each image window, and a more robust evaluation based on statistical analysis and a weighted F-score

Socio-technical lifelogging: deriving design principles for a future proof digital past

Lifelogging is a technically inspired approach that attempts to address the problem of human forgetting by developing systems that ‘record everything’. Uptake of lifelogging systems has generally been disappointing, however. One reason for this lack of uptake is the absence of design principles for developing digital systems to support memory. Synthesising multiple studies, we identify and evaluate 4 new empirically motivated design principles for lifelogging: Selectivity, Embodiment, Synergy and Reminiscence. We first summarise 4 empirical studies that motivate the principles, then describe the evaluation of 4 novel systems built to embody these principles. The design principles were generative, leading to the development of new classes of lifelogging system, as well as providing strategic guidance about how those systems should be built. Evaluations suggest support for Selection and Embodiment principles, but more conceptual and technical work is needed to refine the Synergy and Reminiscence principles

Brain-computer interface for generating personally attractive images

While we instantaneously recognize a face as attractive, it is much harder to explain what exactly defines personal attraction. This suggests that attraction depends on implicit processing of complex, culturally and individually defined features. Generative adversarial neural networks (GANs), which learn to mimic complex data distributions, can potentially model subjective preferences unconstrained by pre-defined model parameterization. Here, we present generative brain-computer interfaces (GBCI), coupling GANs with brain-computer interfaces. GBCI first presents a selection of images and captures personalized attractiveness reactions toward the images via electroencephalography. These reactions are then used to control a GAN model, finding a representation that matches the features constituting an attractive image for an individual. We conducted an experiment (N=30) to validate GBCI using a face-generating GAN and producing images that are hypothesized to be individually attractive. In double-blind evaluation of the GBCI-produced images against matched controls, we found GBCI yielded highly accurate results. Thus, the use of EEG responses to control a GAN presents a valid tool for interactive information-generation. Furthermore, the GBCI-derived images visually replicated known effects from social neuroscience, suggesting that the individually responsive, generative nature of GBCI provides a powerful, new tool in mapping individual differences and visualizing cognitive-affective processing.Peer reviewe

Knowledge-Context in search systems: Toward information-literate actions

In this perspectives paper we define knowledge-context as meta information that searchers use when making sense of information displayed in and accessible from a search engine results page (SERP). We argue that enriching the knowledge-context in SERPs has great potential for facilitating human learning, critical thinking, and creativity by expanding searchers’ information-literate actions such as comparing, evaluating, and differentiating between information sources. Thus it supports the development of learning-centric search systems. Using theories and empirical findings from psychology and the learning sciences, we first discuss general effects of Web search on memory and learning. After reviewing selected research addressing metacognition and self-regulated learning, we discuss design goals for search systems that support metacognitive skills required for long-term learning, creativity, and critical thinking. We then propose that SERPs make both bibliographic and inferential knowledge-context readily accessible to motivate and facilitate information-literate actions for learning and creative endeavors. A brief discussion of related ideas, designs, and prototypes found in prior work follows. We conclude the paper by presenting future research directions and questions on knowledge-context, information-literate actions, and learning-centric search systems.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/148270/1/Smith and Rieh Knowledge-Context in Search Systems CHIIR2019.pd

The Berlin Brain-Computer Interface: Progress Beyond Communication and Control

The combined effect of fundamental results about neurocognitive processes and advancements in decoding mental states from ongoing brain signals has brought forth a whole range of potential neurotechnological applications. In this article, we review our developments in this area and put them into perspective. These examples cover a wide range of maturity levels with respect to their applicability. While we assume we are still a long way away from integrating Brain-Computer Interface (BCI) technology in general interaction with computers, or from implementing neurotechnological measures in safety-critical workplaces, results have already now been obtained involving a BCI as research tool. In this article, we discuss the reasons why, in some of the prospective application domains, considerable effort is still required to make the systems ready to deal with the full complexity of the real world.EC/FP7/611570/EU/Symbiotic Mind Computer Interaction for Information Seeking/MindSeeEC/FP7/625991/EU/Hyperscanning 2.0 Analyses of Multimodal Neuroimaging Data: Concept, Methods and Applications/HYPERSCANNING 2.0DFG, 103586207, GRK 1589: Verarbeitung sensorischer Informationen in neuronalen Systeme