Augmenting spaces and creating interactive experiences using video camera networks

This research addresses the problem of creating interactive experiences to encourage people to explore spaces. Besides the obvious spaces to visit, such as museums or art galleries, spaces that people visit can be, for example, a supermarket or a restaurant. As technology evolves, people become more demanding in the way they use it and expect better forms of interaction with the space that surrounds them. Interaction with the space allows information to be transmitted to the visitors in a friendly way, leading visitors to explore it and gain knowledge. Systems to provide better experiences while exploring spaces demand hardware and software that is not in the reach of every space owner either because of the cost or inconvenience of the installation, that can damage artefacts or the space environment. We propose a system adaptable to the spaces, that uses a video camera network and a wi-fi network present at the space (or that can be installed) to provide means to support interactive experiences using the visitor’s mobile device. The system is composed of an infrastructure (called vuSpot), a language grammar used to describe interactions at a space (called XploreDescription), a visual tool used to design interactive experiences (called XploreBuilder) and a tool used to create interactive experiences (called urSpace). By using XploreBuilder, a tool built of top of vuSpot, a user with little or no experience in programming can define a space and design interactive experiences. This tool generates a description of the space and of the interactions at that space (that complies with the XploreDescription grammar). These descriptions can be given to urSpace, another tool built of top of vuSpot, that creates the interactive experience application. With this system we explore new forms of interaction and use mobile devices and pico projectors to deliver additional information to the users leading to the creation of interactive experiences. The several components are presented as well as the results of the respective user tests, which were positive. The design and implementation becomes cheaper, faster, more flexible and, since it does not depend on the knowledge of a programming language, accessible for the general public.NOVA Laboratory for Computer Science and Informatics (NOVA LINCS), Multimodal Systems, Departamento de Informática (DI), Faculdade de Ciências e Tecnologia (FCT), Universidade Nova de Lisboa (UNL) and Escola Superior de Tecnologia de Setúbal (EST Setúbal), Instituto Politécnico de Setúbal (IPS)

Overcoming Poverty through Digital Inclusion

A growing body of research is showing how digital inclusion can help communities overcome poverty and injustice. The main challenge lies in how best to achieve this goal. The authors argue that digital inclusion must occur in two distinct stages. The first stage is digital literacy, accomplished with the Symbiotic Computer (SC)-smartphones and tablets. The second stage will be professional capacity-building, accomplished with the more traditional Personal Computer (PC)

Control Systems and Ornamental Lighting. A Case Study: Illumination of the Facade of Santiago Hospital in Úbeda (Jaén)

We propose the analysis of the control systems associated with ornamental lighting applied to the architectural and monumental heritage from our teaching experience developed since 2017 at the University of Córdoba—Illumination Faculty of Philosophy of Córdoba, year 2017, and Center of Experimental Culture in the Plaza de la Corredera, year 2018—and the lighting project of the facade of the Hospital de Santiago de Úbeda, Jaén. The DMX system is the one which mostly fulfill the requirements of this kind of lighting. Its implementation is increasingly simple and is associated in many cases with the type of luminaire to be used and the particularity of differential wiring. This solution allows a personalized, creative, and dynamic control and the adaptation of the daily lighting to different events of singular character. It can be used as an instrument of economic revitalization of the tourist and commercial sectors, and of diffusion of the architectural and historical heritage values. The application of the Internet of Things (IoT) and Big Data could allow the creation of a predictive model that helps the spectral design of light sources and an objective tool that contributes to the confluence of art and science

Augmenting conversations through context-aware multimedia retrieval based on speech recognition

Future’s environments will be sensitive and responsive to the presence of people to support them carrying out their everyday life activities, tasks and rituals, in an easy and natural way. Such interactive spaces will use the information and communication technologies to bring the computation into the physical world, in order to enhance ordinary activities of their users. This paper describes a speech-based spoken multimedia retrieval system that can be used to present relevant video-podcast (vodcast) footage, in response to spontaneous speech and conversations during daily life activities. The proposed system allows users to search the spoken content of multimedia files rather than their associated meta-information and let them navigate to the right portion where queried words are spoken by facilitating within-medium searches of multimedia content through a bag-of-words approach. Finally, we have studied the proposed system on different scenarios by using vodcasts in English from various categories, as the targeted multimedia, and discussed how it would enhance people’s everyday life activities by different scenarios including education, entertainment, marketing, news and workplace

End-to-end Projector Photometric Compensation

Projector photometric compensation aims to modify a projector input image such that it can compensate for disturbance from the appearance of projection surface. In this paper, for the first time, we formulate the compensation problem as an end-to-end learning problem and propose a convolutional neural network, named CompenNet, to implicitly learn the complex compensation function. CompenNet consists of a UNet-like backbone network and an autoencoder subnet. Such architecture encourages rich multi-level interactions between the camera-captured projection surface image and the input image, and thus captures both photometric and environment information of the projection surface. In addition, the visual details and interaction information are carried to deeper layers along the multi-level skip convolution layers. The architecture is of particular importance for the projector compensation task, for which only a small training dataset is allowed in practice. Another contribution we make is a novel evaluation benchmark, which is independent of system setup and thus quantitatively verifiable. Such benchmark is not previously available, to our best knowledge, due to the fact that conventional evaluation requests the hardware system to actually project the final results. Our key idea, motivated from our end-to-end problem formulation, is to use a reasonable surrogate to avoid such projection process so as to be setup-independent. Our method is evaluated carefully on the benchmark, and the results show that our end-to-end learning solution outperforms state-of-the-arts both qualitatively and quantitatively by a significant margin.Comment: To appear in the 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Source code and dataset are available at https://github.com/BingyaoHuang/compenne