Proceedings of the 2nd IUI Workshop on Interacting with Smart Objects

These are the Proceedings of the 2nd IUI Workshop on Interacting with Smart Objects. Objects that we use in our everyday life are expanding their restricted interaction capabilities and provide functionalities that go far beyond their original functionality. They feature computing capabilities and are thus able to capture information, process and store it and interact with their environments, turning them into smart objects

Engineering data compendium. Human perception and performance. User's guide

The concept underlying the Engineering Data Compendium was the product of a research and development program (Integrated Perceptual Information for Designers project) aimed at facilitating the application of basic research findings in human performance to the design and military crew systems. The principal objective was to develop a workable strategy for: (1) identifying and distilling information of potential value to system design from the existing research literature, and (2) presenting this technical information in a way that would aid its accessibility, interpretability, and applicability by systems designers. The present four volumes of the Engineering Data Compendium represent the first implementation of this strategy. This is the first volume, the User's Guide, containing a description of the program and instructions for its use

U-DiVE: Design and evaluation of a distributed photorealistic virtual reality environment

This dissertation presents a framework that allows low-cost devices to visualize and interact with photorealistic scenes. To accomplish this task, the framework makes use of Unity’s high-definition rendering pipeline, which has a proprietary Ray Tracing algorithm, and Unity’s streaming package, which allows an application to be streamed within its editor. The framework allows the composition of a realistic scene using a Ray Tracing algorithm, and a virtual reality camera with barrel shaders, to correct the lens distortion needed for the use on an inexpensive cardboard. It also includes a method to collect the mobile device’s spatial orientation through a web browser to control the user’s view, delivered via WebRTC. The proposed framework can produce low-latency, realistic and immersive environments to be accessed through low-cost HMDs and mobile devices. To evaluate the structure, this work includes the verification of the frame rate achieved by the server and mobile device, which should be higher than 30 FPS for a smooth experience. In addition, it discusses whether the overall quality of experience is acceptable by evaluating the delay of image delivery from the server up to the mobile device, in face of user’s movement. Our tests showed that the framework reaches a mean latency around 177 (ms) with household Wi-Fi equipment and a maximum latency variation of 77.9 (ms), among the 8 scenes tested.Esta dissertação apresenta um framework que permite que dispositivos de baixo custo visualizem e interajam com cenas fotorrealísticas. Para realizar essa tarefa, o framework faz uso do pipeline de renderização de alta definição do Unity, que tem um algoritmo de rastreamento de raio proprietário, e o pacote de streaming do Unity, que permite o streaming de um aplicativo em seu editor. O framework permite a composição de uma cena realista usando um algoritmo de Ray Tracing, e uma câmera de realidade virtual com shaders de barril, para corrigir a distorção da lente necessária para usar um cardboard de baixo custo. Inclui também um método para coletar a orientação espacial do dispositivo móvel por meio de um navegador Web para controlar a visão do usuário, entregue via WebRTC. O framework proposto pode produzir ambientes de baixa latência, realistas e imersivos para serem acessados por meio de HMDs e dispositivos móveis de baixo custo. Para avaliar a estrutura, este trabalho considera a verificação da taxa de quadros alcançada pelo servidor e pelo dispositivo móvel, que deve ser superior a 30 FPS para uma experiência fluida. Além disso, discute se a qualidade geral da experiência é aceitável, ao avaliar o atraso da entrega das imagens desde o servidor até o dispositivo móvel, em face da movimentação do usuário. Nossos testes mostraram que o framework atinge uma latência média em torno dos 177 (ms) com equipamentos wi-fi de uso doméstico e uma variação máxima das latências igual a 77.9 (ms), entre as 8 cenas testadas

Machine Learning for Multimedia Communications

Machine learning is revolutionizing the way multimedia information is processed and transmitted to users. After intensive and powerful training, some impressive efficiency/accuracy improvements have been made all over the transmission pipeline. For example, the high model capacity of the learning-based architectures enables us to accurately model the image and video behavior such that tremendous compression gains can be achieved. Similarly, error concealment, streaming strategy or even user perception modeling have widely benefited from the recent learningoriented developments. However, learning-based algorithms often imply drastic changes to the way data are represented or consumed, meaning that the overall pipeline can be affected even though a subpart of it is optimized. In this paper, we review the recent major advances that have been proposed all across the transmission chain, and we discuss their potential impact and the research challenges that they raise

The cognitive neuroscience of visual working memory

Visual working memory allows us to temporarily maintain and manipulate visual information in order to solve a task. The study of the brain mechanisms underlying this function began more than half a century ago, with Scoville and Milner’s (1957) seminal discoveries with amnesic patients. This timely collection of papers brings together diverse perspectives on the cognitive neuroscience of visual working memory from multiple fields that have traditionally been fairly disjointed: human neuroimaging, electrophysiological, behavioural and animal lesion studies, investigating both the developing and the adult brain

The association of early touchscreen media use with the development of visual attention and executive function

Attention plays a pivotal role in information processing by filtering the potential information available based on individual goals, states, and past experiences. Early attention control is thought to underpin and support executive functions (EFs), which in turn are predictive of later behavioural outcomes. The development of attention and EF is partly subject to environmental influences, such as the use of digital media. There is a rapid increase in accessibility and usability of mobile touchscreen devices (i.e. smartphones and tablets) in the family environment, but rigorous scientific research investigating the impact on the developing mind lags behind the widespread usage. To address this, children with different levels of touchscreen use were followed longitudinally at 12 months, 18 months, and 3.5 years, and tested on attention control (bottom-up, and top-down), and EF (updating, shifting, and inhibiting). Children with high touchscreen use were faster on single (i.e. pop-out) visual search, with the amount of concurrent use associated with the speed of bottom-up attention in a linear manner. This saliency bias was repeatedly found on saccadic control tasks, where steady longitudinal high use was associated with a quickening of attention to peripheral salient onsets with a resulting detriment to top-down performance, i.e. disengagement and inhibition of attention. Finally, top-down difficulties were also seen in EF tasks in high users at 3.5 years, particularly in processes of updating and shifting between abstract mental sets. These results point to an influence of touchscreens use on the emerging attention and EF systems, in a way that experience of salient and contingent digital content elicits automatic biases to bottom-up processing, and displaces competency of top-down control and/or increases reliance on stimulus-response pairings. Future studies are needed to demonstrate causality, and to understand long-term trajectories and the interplay between bottom-up and top-down processes over time