Collaborative searching for video using the Físchlár system and a DiamondTouch table

Fischlar DT is one of a family of systems which support interactive searching and browsing through an archive of digital video information. Previous Fischlar systems have used a conventional screen, keyboard and mouse interface, but Fischlar-DT operates with using a horizontal, multiuser, touch sensitive tabletop known as a DiamondTouch. We present the Fischlar-DT system partly from a systems perspective, but mostly in terms of how its design and functionality supports collaborative searching. The contribution of the paper is thus the introduction of Fischlar-DT and a description of how design concerns for supporting collaborative search can be realised on a tabletop interface

Annotations of maps in collaborative work at a distance

This thesis inquires how map annotations can be used to sustain remote collaboration. Maps condense the interplay of space and communication, solving linguistic references by linking conversational content to the actual places to which it refers. This is a mechanism people are accustomed to. When we are face-to-face, we can point to things around us. However, at a distance, we need to recreate a context that can help disambiguate what we mean. A map can help recreate this context. However other technological solutions are required to allow deictic gestures over a shared map when collaborators are not co-located. This mechanism is here termed Explicit Referencing. Several systems that allow sharing maps annotations are reviewed critically. A taxonomy is then proposed to compare their features. Two filed experiments were conducted to investigate the production of collaborative annotations of maps with mobile devices, looking for the reasons why people might want to produce these notes and how they might do so. Both studies led to very disappointing results. The reasons for this failure are attributed to the lack of a critical mass of users (social network), the lack of useful content, and limited social awareness. More importantly, the study identified a compelling effect of the way messages were organized in the tested application, which caused participants to refrain from engaging in content-driven explorations and synchronous discussions. This last qualitative observation was refined in a controlled experiment where remote participants had to solve a problem collaboratively, using chat tools that differed in the way a user could relate an utterance to a shared map. Results indicated that team performance is improved by the Explicit Referencing mechanisms. However, when this is implemented in a way that is detrimental to the linearity of the conversation, resulting in the visual dispersion or scattering of messages, its use has negative consequences for collaborative work at a distance. Additionally, an analysis of the eye movements of the participants over the map helped to ascertain the interplay of deixis and gaze in collaboration. A primary relation was found between the pair's recurrence of eye movements and their task performance. Finally, this thesis presents an algorithm that detects misunderstandings in collaborative work at a distance. It analyses the movements of collaborators' eyes over the shared map, their utterances containing references to this workspace, and the availability of "remote" deictic gestures. The algorithm associates the distance between the gazes of the emitter and gazes of the receiver of a message with the probability that the recipient did not understand the message

Haptic augmentation of the cursor: transforming virtual actions into physical actions

This thesis demonstrates, through the exploration of two very different examples, the general claim that haptic feedback relating to a user's representation in a computer system (typically a cursor) can lead to increases in objective performance and subjective experience. Design guidelines covering each of these two topics are also presented, to ensure that the research described here can be readily adopted by other researchers, designers and system developers. The first topic to be investigated was desktop user interfaces. This thesis describes the design of a variety of different forms of haptic feedback for use with number of different Graphical User Interface (GUI) widgets, or widget groups. Two empirical evaluations of these designs are also described in some depth. The results of these studies indicate that although haptic feedback can provide improvements in objective performance, it can also reduce performance and increase subjective workload if inappropriately applied. From these results, and from the previous literature, detailed guidelines were drawn up covering the addition of haptic feedback to GUIs. The goal of these guidelines is to support the creation of performance-enhancing haptic feedback. The second topic examined was communication in interactive collaborative systems. The design of a suite of haptic communication is presented in detail, before two studies investigating different aspects of its use. The first study focuses on the subjective influence of the haptic communication as a whole, while the second is a more thorough look at one particular form of the feedback and includes objective measurements. The combined results of these studies indicate that haptic feedback has a valuable potential for increasing the quality of a user's subjective experience. Observations from these studies also reveal insights into the role of haptic feedback in communication. A set of guidelines summing up this research and the previous literature relevant to this topic are then presented. As research on this domain is in its infancy, the goal of these guidelines is to concisely present the main issues and potential benefits that respectively restrict and drive this work

Consistency Algorithms and Protocols for Distributed Interactive Applications

The Internet has a major impact not only on how people retrieve information but also on how they communicate. Distributed interactive applications support the communication and collaboration of people through the sharing and manipulation of rich multimedia content via the Internet. Aside from shared text editors, meeting support systems, and distributed virtual environments, shared whiteboards are a prominent example of distributed interactive applications. They allow the presentation and joint editing of documents in video conferencing scenarios. The design of such a shared whiteboard application, the multimedia lecture board (mlb), is a main contribution of this thesis. Like many other distributed interactive applications, the mlb has a replicated architecture where each user runs an instance of the application. This has the distinct advantage that the application can be deployed in a lightweight fashion, without relying on a supporting server infrastructure. But at the same time, this peer-to-peer architecture raises a number of challenging problems: First, application data needs to be distributed among all instances. For this purpose, we present the network protocol RTP/I for the standardized communication of distributed interactive applications, and a novel application-level multicast protocol that realizes efficient group communication while taking application-level knowledge into account. Second, consistency control mechanisms are required to keep the replicated application data synchronized. We present the consistency control algorithms “local lag”, “Timewarp”, and “state request”, show how they can be combined, and discuss how to provide visual feedback so that the session members are able to handle conflicting actions. Finally, late-joining participants need to be initialized with the current application state before they are able to participate in a collaborative session. We propose a novel late-join algorithm, which is both flexible and scalable. All algorithms and protocols presented in this dissertation solve the aforementioned problems in a generic way. We demonstrate how they can be employed for the mlb as well as for other distributed interactive applications

Group reaching over digital tabletops with digital arm embodiments

In almost all collaborative tabletop tasks, groups require coordinated access to the shared objects on the table’s surface. The physical social norms of close-proximity interactions built up over years of interacting around other physical bodies cause people to avoid interfering with other people (e.g., avoiding grabbing the same object simultaneously). However, some digital tabletop situations require the use of indirect input (e.g., when using mice, and when supporting remote users). With indirect input, people are no longer physically embodied during their reaching gestures, so most systems provide digital embodiments – visual representations of each person – to provide feedback to both the person who is reaching and to the other group members. Tabletop arm embodiments have been shown to better support group interactions than simple visual designs, providing awareness of actions to the group. However, researchers and digital tabletop designers know little of how the design of digital arm embodiments affects the fundamental group tabletop interaction of reaching for objects. Therefore, in this thesis, we evaluate how people coordinate their interactions over digital tabletops when using different types of embodiments. Specifically, in a series of studies, we investigate how the visual design (what they look like) and interaction design (how they work) of digital arm embodiments affects a group’s coordinative behaviours in an open- ended parallel tabletop task. We evaluated visual factors of size, transparency, and realism (through pictures and videos of physical arms), as well as interaction factors of input and augmentations (feedback of interactions), in both a co-located and distributed environment. We found that the visual design had little effect on a group’s ability to coordinate access to shared tabletop items, that embodiment augmentations are useful to support group coordinative actions, and that there are large differences when the person is not physically co-present. Our results demonstrate an initial exploration into the design of digital arm embodiments, providing design guidelines for future researchers and designers to use when designing the next generation of shared digital spaces

Development of virtual network computing (VNC) environment for networking and enhancing user experience

Virtual Network Computing (VNC) is a thin client developed by Real VNC Ltd, Formerly of Olivetti Research Ltd/AT&T labs Cambridge and can be used as a collaborative environment, therefore it has been chosen as the basis of this research study. The purpose of this thesis is to investigate and develop a VNC based environment over the network and to improve the users’ Quality of Experience (QoE) of using VNC between networked groups by the incorporation of videoconferencing with VNC and enhancing QoE in Mobile environments where the network status is far from ideal and is prone to disconnection. This thesis investigates the operation of VNC in different environments and scenarios such as wireless environments by investigating user and device mobility and ways to sustain their seamless connection when in motion. As part of the study I also researched all groups that implement VNC like universities, research groups and laboratories and virtual laboratories. In addition to that I identified the successful features and security measures in VNC in order to create a secure environment. This was achieved by pinpointing the points of strength and weakness in VNC as opposed to popular thin clients and remote control applications and analysing VNC according to conforming to several security measures. Furthermore, it is reasonable to say that the success of any scheme that attempts to deliver desirable levels of Quality of Service (QoS) of an effective application for the future Internet must be based, not only on the progress of technology, but on usersʹ requirements. For instance, a collaborative environment has not yet reached the desired expectation of its users since it is not capable of handling any unexpected events which can result from a sudden disconnection of a nomadic user engaged in an ongoing collaborative session; this is consequently associated with breaking the social dynamics of the group collaborating in the session. Therefore, I have concluded that knowing the social dynamics of application’s users as a group and their requirements and expectations of a successful experience can lead an application designer to exploit technology to autonomously support the initiating and maintaining of social interaction. Moreover, I was able to successfully develop a VNC based environment for networked groups that facilitates the administration of different remote VNC sessions. In addition to a prototype that uses videoconferencing in parallel to VNC to provide a better user’s QoE of VNC. The last part of the thesis was concerned with designing a framework to improve and assess QoE of all users in a collaborative environment where it can be especially applied in the presence of nomadic clients with their much frequent disconnections. I have designed a conceptual algorithm called Improved Collaborative Quality of Experience (IC‐QoE), an algorithm that aims to eliminate frustration and improve QoE of users in a collaborative session in the case of disconnections and examined its use and benefits in real world scenarios such as research teams and implemented a prototype to present the concepts of this algorithm. Finally, I have designed a framework to suggest ways to evaluate this algorithm

Optimizing transmission protocols for enhancement of quality of service in telemedical realtime applications

In der Dissertation mit dem Titel „Optimierung von Übertragungsprotokollen zur Verbesserung der Dienstgüte in telemedizinischen Echtzeitanwendungen“ geht es um die Entwicklung einer Protokollmodifikation für Multipath-TCP zur besseren Unterstützung von telemedizinischen Echtzeitanwendungen über das Internet in einem Szenario zwischen global verteilten Standorten. Das Ziel von redundantem Multipath-TCP (rMPTCP) ist es, mehrere Verbindungen gleichzeitig zu nutzen, um mithilfe von Redundanz Verzögerungsspitzen und Datenverluste auszugleichen und somit die Dienstgüte der Übertragung zu verbessern. Hierbei passt sich das Protokoll den aktuellen Gegebenheiten der Datenverbindung adaptiv an, indem es Redundanz, Leitungsqualität, benötigte Datenübertragungsrate sowie den durch das Netzwerk angebotenen Datendurchsatz in Relation setzt. Anwendungen in der Telemedizin unterscheiden sich in ihren kommunikativen und interaktiven Ausprägungen und damit in ihren Dienstgüteanforderungen. Zu diesem Zweck werden grundlegende Anwendungen in der Telemedizin sowie Spezifizierungen der Dienstgüteanforderungen der anfallenden Datenströme behandelt und klassifiziert. Dies geschieht in Hinblick auf ein zwischen der Universität Duisburg-Essen (UDE) und der Universiti Kebangsaan Malaysia (UKM) laufendes Forschungsszenario. Darauf folgt eine Darlegung von Dienstgüte-Mechanismen im Internet. Darin werden die elementaren Funktionsweisen sowie Möglichkeiten diese zu verbessern beschrieben. Die Übertragungsstrecke zwischen den beiden Universitäten wird entsprechend des Basisszenarios in Hinblick auf verschiedene Dienstgüteparameter mit entsprechenden Messwerkzeugen soweit ausgewertet, dass Gegebenheiten und Probleme identifiziert werden können. Eine Evaluierung der verschiedenen verfügbaren Verbindungen zwischen UDE und UKM dient der Ermittlung einer kombinierten Nutzungsweise und den Möglichkeiten bei einer Mehrfachverbindung. Die Modellierung und Entwicklung der Protokollmodifikation wird unter den vorher hergeleiteten Anforderungen durchgeführt. Es werden die grundsätzlichen mathematischen Zusammenhänge diskutiert und eine Einführung in die Funktionalitäten des Protokolls gegeben. Die Eigenschaften und Funktionen des neuen Protokolls werden modelliert und zusätzliche Hilfsmittel, die für die Anwendung innerhalb des Szenarios benötigt werden entwickelt. Die Funktionalitäten werden in praktischen Versuchen ausgewertet und eine abschließende Beurteilung diskutiert. Das Ergebnis dieser Dissertation ist die Entwicklung einer internetkompatiblen redundanten Protokollerweiterung für Multipath-TCP, die in der Lage ist, sich mithilfe verschiedener Algorithmen auf Situationen im Netzwerk anzupassen und verschiedene Maßnahmen bei Störungen zu ergreifen. Die Protokollerweiterung ist in der Lage, eine Dienstgüteverbesserung in Hinblick auf Verzögerungen sowie Verzögerungsvarianzen für Anwendungen mit „Nahe-Echtzeit“-Anforderungen zu erreichen.In the dissertation entitled "Optimization of Transmission Protocols to Improve Quality of Service in Telemedicine Real-Time Applications", the aim is to develop a protocol modification for Multipath-TCP to improve the support for telemedical real-time applications over the Internet in a scenario between globally distributed locations. The goal of redundant Multipath TCP (rMPTCP) is to use multiple connections at the same time to compensate for delay and data losses by means of redundancy and thus to improve the quality of service of the transmission. The protocol adapts to the current circumstances of the data connection by correlating redundancy, connection quality, required data transmission rate as well as the data throughput offered by the network. Applications in telemedicine differ in their communicative and interactive manifestations and thus in their quality of service requirements. For this purpose, basic applications in telemedicine as well as specifications of the quality requirements of the resulting data streams are treated and classified. This is done with regard to a research scenario running between the University of Duisburg-Essen (UDE) and the Universiti Kebangsaan Malaysia (UKM). It is followed by a presentation of quality of service mechanisms on the Internet. The basic functions as well as ways to improve them are described. The transmission distance between the two universities is evaluated according to the basic scenario with regard to different quality of service parameters with appropriate measuring tools in a way that conditions and problems can be identified. An evaluation of the various available connections between UDE and UKM is used to determine a combined usage and the possibilities for a multiple connection. The modeling and development of the protocol modification is performed under the previously-derived requirements. The basic mathematical connections are discussed and an introduction to the functionalities of the protocol is given. The properties and functions of the new protocol are modeled and additional tools developed for utilization within the scenario are designed. The functionalities are evaluated in practical tests and a final assessment is discussed. The result of this dissertation is the development of an Internet-compatible redundant protocol extension for multipath TCP, which is able to adapt to situations in the network by means of different algorithms and to take various measures in case of disturbances. The protocol extension is capable of achieving an improvement in service quality with regard to delays as well as delay variances for applications with "near real-time" requirements

Interacção por toque em múltiplas superfícies

Tese de mestrado, Engenharia Informática (Arquitectura, Sistemas e Redes de Computadores), Universidade de Lisboa, Faculdade de Ciências, 2009Actualmente, com o preço dos pixéis a diminuir, os ecrãs de computador tendem a aumentar de tamanho. Os ecrãs de parede e outras superfícies de interacção de grande dimensão são agora uma opção para muitos utilizadores. Esta tendência levanta várias questões a serem investigadas na área da interacção pessoa-máquina. A aproximação simplista de transferir os principais conceitos de interacção do paradigma clássico WIMP (Windows, Icon, Menu, Pointer / Janela, Ícone, Menu, Dispositivo Apontador), baseado nos dispositivos tradicionais de entrada, rato e teclado, rapidamente conduz a problemas inesperados. Nos últimos anos também se assistiu à emergência dos primeiros produtos comerciais a suportarem interacções multi-toque. É expectável que a tecnologia do toque se torne brevemente standard, o que já é visível em alguns mercados específicos, como o dos telemóveis. Se juntarmos as possibilidades criadas pela recente “revolução táctil” e a transição a que assistimos nos últimos anos para ecrãs de grande dimensão, estamos agora em condições de explorar como a interacção gestual pode contribuir para minimizar os problemas com o paradigma clássico WIMP em ecrãs de grande dimensão. Neste trabalho é explorado o campo da interacção gestual em ecrãs de grande dimensão. Foram conduzidos vários estudos, onde os utilizadores experimentaram interacção gestual, em várias aplicações adequadas para ecrãs de grande dimensão. Os resultados demonstram que a manipulação directa através de interacção gestual é apelativa aos utilizadores para alguns tipos de aplicações e acções, enquanto para outros tipos, os gestos não são a modalidade preferida de interacção. Posteriormente, introduziu-se o uso da interacção gestual para cenários cooperativos, discutindo a sua adequação a algumas tarefas, e a forma como os utilizadores decidem cooperativamente que tarefas realizar baseados nas modalidades de entrada disponíveis e nas características das tarefas.Nowadays, with pixels getting cheaper, computer displays tend toward larger sizes. Wall sized screens and other large interaction surfaces are now an option for many users and this trend raises a number of issues to be researched in the user interface area. The simplistic approach of transferring the main interaction concepts of the classic WIMP (Window, Icon, Menu, Pointer) design paradigm, based on the traditional mouse and keyboard input devices, quickly led to unexpected problems. In recent years we also witnessed a revolution with the emergence of the first commercial products supporting multi-touch interaction. It is expected that the use of touch technology will soon become standard, and this is already visible in some specific markets, such as mobile phones. If we put together the possibilities opened up by the recent “touch revolution” and the transition we have been witnessing for the past few years to large screen displays, we are now able to explore how the use of gestural interaction can contribute to overcome the problems with the classical WIMP paradigm in large screen displays. In this work we explore the field of gestural interaction on large screen displays, conducting several studies where users experience gestural interaction in various applications suited for large displays. Results show how direct manipulation through gestural interaction appeals to users for some types of applications and actions, while demonstrating that for other types, gestures should not be the preferred interaction modality. Afterward, we introduce the use of gestural interaction for cooperative scenarios, discussing how it is more suited for some tasks, and hypothesizing on how users cooperatively decide on which tasks to perform based on the available input modalities and task characteristics