Video interaction using pen-based technology

Dissertação para obtenção do Grau de Doutor em InformáticaVideo can be considered one of the most complete and complex media and its manipulating is still a difficult and tedious task. This research applies pen-based technology to video manipulation, with the goal to improve this interaction. Even though the human familiarity with pen-based devices, how they can be used on video interaction, in order to improve it, making it more natural and at the same time fostering the user’s creativity is an open question. Two types of interaction with video were considered in this work: video annotation and video editing. Each interaction type allows the study of one of the interaction modes of using pen-based technology: indirectly, through digital ink, or directly, trough pen gestures or pressure. This research contributes with two approaches for pen-based video interaction: pen-based video annotations and video as ink. The first uses pen-based annotations combined with motion tracking algorithms, in order to augment video content with sketches or handwritten notes. It aims to study how pen-based technology can be used to annotate a moving objects and how to maintain the association between a pen-based annotations and the annotated moving object The second concept replaces digital ink by video content, studding how pen gestures and pressure can be used on video editing and what kind of changes are needed in the interface, in order to provide a more familiar and creative interaction in this usage context.This work was partially funded by the UTAustin-Portugal, Digital Media, Program (Ph.D. grant: SFRH/BD/42662/2007 - FCT/MCTES); by the HP Technology for Teaching Grant Initiative 2006; by the project "TKB - A Transmedia Knowledge Base for contemporary dance" (PTDC/EAT/AVP/098220/2008 funded by FCT/MCTES); and by CITI/DI/FCT/UNL (PEst-OE/EEI/UI0527/2011

Interactive Remote Collaboration Using Augmented Reality

With the widespread deployment of fast data connections and availability of a variety of sensors for different modalities, the potential of remote collaboration has greatly increased. While the now ubiquitous video conferencing applications take advantage of some of these capabilities, the use of video between remote users is limited to passively watching disjoint video feeds and provides no means for interaction with the remote environment. However, collaboration often involves sharing, exploring, referencing, or even manipulating the physical world, and thus tools should provide support for these interactions.We suggest that augmented reality is an intuitive and user-friendly paradigm to communicate information about the physical environment, and that integration of computer vision and augmented reality facilitates more immersive and more direct interaction with the remote environment than what is possible with today's tools.In this dissertation, we present contributions to realizing this vision on several levels. First, we describe a conceptual framework for unobtrusive mobile video-mediated communication in which the remote user can explore the live scene independent of the local user's current camera movement, and can communicate information by creating spatial annotations that are immediately visible to the local user in augmented reality. Second, we describe the design and implementation of several, increasingly more flexible and immersive user interfaces and system prototypes that implement this concept. Our systems do not require any preparation or instrumentation of the environment; instead, the physical scene is tracked and modeled incrementally using monocular computer vision. The emerging model then supports anchoring of annotations, virtual navigation, and synthesis of novel views of the scene. Third, we describe the design, execution and analysis of three user studies comparing our prototype implementations with more conventional interfaces and/or evaluating specific design elements. Study participants overwhelmingly preferred our technology, and their task performance was significantly better compared with a video-only interface, though no task performance difference was observed compared with a ``static marker'' interface. Last, we address a particular technical limitation of current monocular tracking and mapping systems which was found to be impeding and present a conceptual solution; namely, we describe a concept and proof-of-concept implementation for automatic model selection which allows tracking and modeling to cope with both parallax-inducing and rotation-only camera movements.We suggest that our results demonstrate the maturity and usability of our systems, and, more importantly, the potential of our approach to improve video-mediated communication and broaden its applicability

Conceitos e métodos para apoio ao desenvolvimento e avaliação de colaboração remota utilizando realidade aumentada

Remote Collaboration using Augmented Reality (AR) shows great potential to establish a common ground in physically distributed scenarios where team-members need to achieve a shared goal. However, most research efforts in this field have been devoted to experiment with the enabling technology and propose methods to support its development. As the field evolves, evaluation and characterization of the collaborative process become an essential, but difficult endeavor, to better understand the contributions of AR. In this thesis, we conducted a critical analysis to identify the main limitations and opportunities of the field, while situating its maturity and proposing a roadmap of important research actions. Next, a human-centered design methodology was adopted, involving industrial partners to probe how AR could support their needs during remote maintenance. These outcomes were combined with literature methods into an AR-prototype and its evaluation was performed through a user study. From this, it became clear the necessity to perform a deep reflection in order to better understand the dimensions that influence and must/should be considered in Collaborative AR. Hence, a conceptual model and a humancentered taxonomy were proposed to foster systematization of perspectives. Based on the model proposed, an evaluation framework for contextualized data gathering and analysis was developed, allowing support the design and performance of distributed evaluations in a more informed and complete manner. To instantiate this vision, the CAPTURE toolkit was created, providing an additional perspective based on selected dimensions of collaboration and pre-defined measurements to obtain “in situ” data about them, which can be analyzed using an integrated visualization dashboard. The toolkit successfully supported evaluations of several team-members during tasks of remote maintenance mediated by AR. Thus, showing its versatility and potential in eliciting a comprehensive characterization of the added value of AR in real-life situations, establishing itself as a generalpurpose solution, potentially applicable to a wider range of collaborative scenarios.Colaboração Remota utilizando Realidade Aumentada (RA) apresenta um enorme potencial para estabelecer um entendimento comum em cenários onde membros de uma equipa fisicamente distribuídos precisam de atingir um objetivo comum. No entanto, a maioria dos esforços de investigação tem-se focado nos aspetos tecnológicos, em fazer experiências e propor métodos para apoiar seu desenvolvimento. À medida que a área evolui, a avaliação e caracterização do processo colaborativo tornam-se um esforço essencial, mas difícil, para compreender as contribuições da RA. Nesta dissertação, realizámos uma análise crítica para identificar as principais limitações e oportunidades da área, ao mesmo tempo em que situámos a sua maturidade e propomos um mapa com direções de investigação importantes. De seguida, foi adotada uma metodologia de Design Centrado no Humano, envolvendo parceiros industriais de forma a compreender como a RA poderia responder às suas necessidades em manutenção remota. Estes resultados foram combinados com métodos da literatura num protótipo de RA e a sua avaliação foi realizada com um caso de estudo. Ficou então clara a necessidade de realizar uma reflexão profunda para melhor compreender as dimensões que influenciam e devem ser consideradas na RA Colaborativa. Foram então propostos um modelo conceptual e uma taxonomia centrada no ser humano para promover a sistematização de perspetivas. Com base no modelo proposto, foi desenvolvido um framework de avaliação para recolha e análise de dados contextualizados, permitindo apoiar o desenho e a realização de avaliações distribuídas de forma mais informada e completa. Para instanciar esta visão, o CAPTURE toolkit foi criado, fornecendo uma perspetiva adicional com base em dimensões de colaboração e medidas predefinidas para obter dados in situ, que podem ser analisados utilizando o painel de visualização integrado. O toolkit permitiu avaliar com sucesso vários colaboradores durante a realização de tarefas de manutenção remota apoiada por RA, permitindo mostrar a sua versatilidade e potencial em obter uma caracterização abrangente do valor acrescentado da RA em situações da vida real. Sendo assim, estabelece-se como uma solução genérica, potencialmente aplicável a uma gama diversificada de cenários colaborativos.Programa Doutoral em Engenharia Informátic

Best Practices for Conducting Virtual Focus Groups

This paper briefly traces focus groups from its origins, through its development from in-person to virtual platforms, which are more commonplace today. The terms online and virtual are used interchangeably in the research literature. In this paper, the term virtual is used to describe focus groups that are face-to-face using technology that incorporate audiovisual features. This paper begins with a description of the origins and purpose of focus groups, the purpose, and how they have evolved to include synchronous, and asynchronous online text-based, and virtual face-to-face focus groups. Next, the paper outlines the role of the facilitator. Comparisons of best practices for in-person and virtual focus groups, including facilitator cues and facilitation best practices, techniques, and strategies; best practices and challenges of conducting virtual focus groups; and video-conferencing tools are also explored. A brief discussion regarding collecting and analyzing the data is also included

Perspectives in visual imaging for marine biology and ecology: from acquisition to understanding

Durden J, Schoening T, Althaus F, et al. Perspectives in Visual Imaging for Marine Biology and Ecology: From Acquisition to Understanding. In: Hughes RN, Hughes DJ, Smith IP, Dale AC, eds. Oceanography and Marine Biology: An Annual Review. 54. Boca Raton: CRC Press; 2016: 1-72

The social AR continuum : wearable AR for sharing social experiences.

The primary goal of the work reported in this thesis is to explore, develop and evaluate novel interaction techniques for Augmented Reality (AR) wearable head- sets, by focusing on how they can be used to share social experiences with family and friends. AR has the potential to provide an intuitive and natural approach to sharing our social experiences in life with others while being co-present. In order to better visualise and interact with social networks on wearable AR devices, we intro- duce the concept of the "Social AR Continuum", which describes the space of sharing experiences in AR across various axes. We discuss the advantages and limitations of various implementations and techniques of shared social experiences on wearable AR. Based on Human-Computer Interaction methodologies, we conducted user studies to evaluate user presences and system usability of our implementations for visualising, sharing and interacting with social experiences on AR headsets. The work focused on the essential Social AR Continuum dimensions of representing contacts, sharing data and interactions, developed user interfaces on these dimensions and evaluated them using user studies. Our results show that using AR to share social experiences can increase users’ social presence. The results are summarised as design recommendations to help interface designers better design shared social experiences on wearable AR systems

Developing a collaborative MOOC learning environment utilizing video sharing with discussion summarization as added-value

With the fast-growing Massive Open Online Courses (MOOC) community and the increase in the number of Learning Management Systems (LMSs) available online, the amount of shared information is massive. Current LMS - in particular MOOC providers - offer many advanced content delivery techniques: interactive video, active retrieval practices, and quizzes to enhance the pedagogical process. The main knowledge creation assets within MOOCs are encapsulated in other tools such as discussion forums, blogs, and wikis. Although these tools exist as separate entities within the platform, they still follow traditional techniques. We believe these tools need to be fully integrated to the main content and encourage spontaneous collaboration. From my experience with some MOOCs, the amount of collaboration and information-sharing is still overwhelming due to the massive number of participants and the limited range of collaborative tools. However, most of the shared information could be redundant or irrelevant. This information must be processed in order to provide the most concise knowledge. Therefore, we need to summarize this information from the discussions, blogs, and wikis and include the most relevant data in the course content. This thesis addresses this shortcoming by suggesting a new system with two primary components to accomplish this task. In the first component, we link the discussion tools to the main course content. Then, in the second component, we apply Natural Language Processing (NLP) techniques to present a summary of all shared content. We use techniques such as Term Frequency - Inverse Document Frequency (TF-IDF), stemming algorithm, Vector Space Model (VSM), and Cosine Similarity to rank the sentences. We then tune the TF-IDF values and boost the sentence ranks using the main content by delegating the first component’s features. The next step involves choosing the most relevant sentence to build our summary. Finally, we evaluate our result using Recall-Oriented Understudy for Gisting Evaluation (ROUGE) system, which compares our automated summary to human extracted summaries. These results demonstrate that we can achieve high improvement summar y compared to the baseline and other similar techniques

Privacy-Respecting Smart Video Surveillance Based on Usage Control Enforcement

This research introduces a conceptual framework for enforcing privacy-related restrictions in smart video surveillance systems based on danger levels and incident types to be handled. It increases the selectivity of surveillance by restricting data processing to individuals associated to incidents under investigation. Constraints are enforced by usage control, which is instantiated for video surveillance for the first time and enables tailoring such systems to comply with data protection law

Parkinson's Disease Management through ICT

Parkinson's Disease (PD) is a neurodegenerative disorder that manifests with motor and non-motor symptoms. PD treatment is symptomatic and tries to alleviate the associated symptoms through an adjustment of the medication. As the disease is evolving and this evolution is patient specific, it could be very difficult to properly manage the disease.The current available technology (electronics, communication, computing, etc.), correctly combined with wearables, can be of great use for obtaining and processing useful information for both clinicians and patients allowing them to become actively involved in their condition.Parkinson's Disease Management through ICT: The REMPARK Approach presents the work done, main results and conclusions of the REMPARK project (2011 – 2015) funded by the European Union under contract FP7-ICT-2011-7-287677. REMPARK system was proposed and developed as a real Personal Health Device for the Remote and Autonomous Management of Parkinson’s Disease, composed of different levels of interaction with the patient, clinician and carers, and integrating a set of interconnected sub-systems: sensor, auditory cueing, Smartphone and server. The sensor subsystem, using embedded algorithmics, is able to detect the motor symptoms associated with PD in real time. This information, sent through the Smartphone to the REMPARK server, is used for an efficient management of the disease

Designing to Support Workspace Awareness in Remote Collaboration using 2D Interactive Surfaces

Increasing distributions of the global workforce are leading to collaborative workamong remote coworkers. The emergence of such remote collaborations is essentiallysupported by technology advancements of screen-based devices ranging from tabletor laptop to large displays. However, these devices, especially personal and mobilecomputers, still suffer from certain limitations caused by their form factors, that hinder supporting workspace awareness through non-verbal communication suchas bodily gestures or gaze. This thesis thus aims to design novel interfaces andinteraction techniques to improve remote coworkers’ workspace awareness throughsuch non-verbal cues using 2D interactive surfaces.The thesis starts off by exploring how visual cues support workspace awareness infacilitated brainstorming of hybrid teams of co-located and remote coworkers. Basedon insights from this exploration, the thesis introduces three interfaces for mobiledevices that help users maintain and convey their workspace awareness with their coworkers. The first interface is a virtual environment that allows a remote person to effectively maintain his/her awareness of his/her co-located collaborators’ activities while interacting with the shared workspace. To help a person better express his/her hand gestures in remote collaboration using a mobile device, the second interfacepresents a lightweight add-on for capturing hand images on and above the device’sscreen; and overlaying them on collaborators’ device to improve their workspace awareness. The third interface strategically leverages the entire screen space of aconventional laptop to better convey a remote person’s gaze to his/her co-locatedcollaborators. Building on the top of these three interfaces, the thesis envisions an interface that supports a person using a mobile device to effectively collaborate with remote coworkers working with a large display.Together, these interfaces demonstrate the possibilities to innovate on commodity devices to offer richer non-verbal communication and better support workspace awareness in remote collaboration