An Information-Theoretic Framework for Consistency Maintenance in Distributed Interactive Applications

Distributed Interactive Applications (DIAs) enable geographically dispersed users to interact with each other in a virtual environment. A key factor to the success of a DIA is the maintenance of a consistent view of the shared virtual world for all the participants. However, maintaining consistent states in DIAs is difficult under real networks. State changes communicated by messages over such networks suffer latency leading to inconsistency across the application. Predictive Contract Mechanisms (PCMs) combat this problem through reducing the number of messages transmitted in return for perceptually tolerable inconsistency. This thesis examines the operation of PCMs using concepts and methods derived from information theory. This information theory perspective results in a novel information model of PCMs that quantifies and analyzes the efficiency of such methods in communicating the reduced state information, and a new adaptive multiple-model-based framework for improving consistency in DIAs. The first part of this thesis introduces information measurements of user behavior in DIAs and formalizes the information model for PCM operation. In presenting the information model, the statistical dependence in the entity state, which makes using extrapolation models to predict future user behavior possible, is evaluated. The efficiency of a PCM to exploit such predictability to reduce the amount of network resources required to maintain consistency is also investigated. It is demonstrated that from the information theory perspective, PCMs can be interpreted as a form of information reduction and compression. The second part of this thesis proposes an Information-Based Dynamic Extrapolation Model for dynamically selecting between extrapolation algorithms based on information evaluation and inferred network conditions. This model adapts PCM configurations to both user behavior and network conditions, and makes the most information-efficient use of the available network resources. In doing so, it improves PCM performance and consistency in DIAs

Reducing the effect of network delay on tightly-coupled interaction

Tightly-coupled interaction is shared work in which each person’s actions immediately and continuously influence the actions of others. Tightly-coupled interaction is a hallmark of expert behaviour in face-to-face activity, but becomes extremely difficult to accomplish in distributed groupware. The main cause of this difficulty is network delay – even amounts as small as 100ms – that disrupts people’s ability to synchronize their actions with another person. To reduce the effects of delay on tightly-coupled interaction, I introduce a new technique called Feedback-Feedthrough Synchronization (FFS). FFS causes visual feedback from an action to occur at approximately the same time for both the local and the remote person, preventing one person from getting ahead of the other in the coordinated interaction. I tested the effects of FFS on group performance in several delay conditions, and my study showed that FFS substantially improved users’ performance: accuracy was significantly improved at all levels of delay, and without noticeable increase in perceived effort or frustration. Techniques like FFS that support the requirements of tightly-coupled interaction provide new means for improving the usability of groupware that operates on real-world networks

Real-Time Markerless Tracking the Human Hands for 3D Interaction

This thesis presents methods for enabling suitable human computer interaction using only movements of the bare human hands in free space. This kind of interaction is natural and intuitive, particularly because actions familiar to our everyday life can be reflected. Furthermore, the input is contact-free which is of great advantage e.g. in medical applications due to hygiene factors. For enabling the translation of hand movements to control signals an automatic method for tracking the pose and/or posture of the hand is needed. In this context the simultaneous recognition of both hands is desirable to allow for more natural input. The first contribution of this thesis is a novel video-based method for real-time detection of the positions and orientations of both bare human hands in four different predefined postures, respectively. Based on such a system novel interaction interfaces can be developed. However, the design of such interfaces is a non-trivial task. Additionally, the development of novel interaction techniques is often mandatory in order to enable the design of efficient and easily operable interfaces. To this end, several novel interaction techniques are presented and investigated in this thesis, which solve existing problems and substantially improve the applicability of such a new device. These techniques are not restricted to this input instrument and can also be employed to improve the handling of other interaction devices. Finally, several new interaction interfaces are described and analyzed to demonstrate possible applications in specific interaction scenarios.Markerlose Verfolgung der menschlichen Hände in Echtzeit für 3D Interaktion In der vorliegenden Arbeit werden Verfahren dargestellt, die sinnvolle Mensch- Maschine-Interaktionen nur durch Bewegungen der bloßen Hände in freiem Raum ermöglichen. Solche "natürlichen" Interaktionen haben den besonderen Vorteil, dass alltägliche und vertraute Handlungen in die virtuelle Umgebung übertragen werden können. Außerdem werden auf diese Art berührungslose Eingaben ermöglicht, nützlich z.B. wegen hygienischer Aspekte im medizinischen Bereich. Um Handbewegungen in Steuersignale umsetzen zu können, ist zunächst ein automatisches Verfahren zur Erkennung der Lage und/oder der Art der mit der Hand gebildeten Geste notwendig. Dabei ist die gleichzeitige Erfassung beider Hände wünschenswert, um die Eingaben möglichst natürlich gestalten zu können. Der erste Beitrag dieser Arbeit besteht aus einer neuen videobasierten Methode zur unmittelbaren Erkennung der Positionen und Orientierungen beider Hände in jeweils vier verschiedenen, vordefinierten Gesten. Basierend auf einem solchen Verfahren können neuartige Interaktionsschnittstellen entwickelt werden. Allerdings ist die Ausgestaltung solcher Schnittstellen keinesfalls trivial. Im Gegenteil ist bei einer neuen Art der Interaktion meist sogar die Entwicklung neuer Interaktionstechniken erforderlich, damit überhaupt effiziente und gut bedienbare Schnittstellen konzipiert werden können. Aus diesem Grund wurden in dieser Arbeit einige neue Interaktionstechniken entwickelt und untersucht, die vorhandene Probleme beheben und die Anwendbarkeit eines solchen Eingabeinstruments für bestimmte Arten der Interaktion verbessern oder überhaupt erst ermöglichen. Diese Techniken sind nicht auf dieses Eingabeinstrument beschränkt und können durchaus auch die Handhabung anderer Eingabegeräte verbessern. Des Weiteren werden mehrere neue Interaktionsschnittstellen präsentiert, die den möglichen Einsatz bloßhändiger Interaktion in verschiedenen, typischen Anwendungsgebieten veranschaulichen

Physically Interacting With Four Dimensions

Thesis (Ph.D.) - Indiana University, Computer Sciences, 2009People have long been fascinated with understanding the fourth dimension. While making pictures of 4D objects by projecting them to 3D can help reveal basic geometric features, 3D graphics images by themselves are of limited value. For example, just as 2D shadows of 3D curves may have lines crossing one another in the shadow, 3D graphics projections of smooth 4D topological surfaces can be interrupted where one surface intersects another. The research presented here creates physically realistic models for simple interactions with objects and materials in a virtual 4D world. We provide methods for the construction, multimodal exploration, and interactive manipulation of a wide variety of 4D objects. One basic achievement of this research is to exploit the free motion of a computer-based haptic probe to support a continuous motion that follows the \emph{local continuity\/} of a 4D surface, allowing collision-free exploration in the 3D projection. In 3D, this interactive probe follows the full local continuity of the surface as though we were in fact \emph{physically touching\/} the actual static 4D object. Our next contribution is to support dynamic 4D objects that can move, deform, and collide with other objects as well as with themselves. By combining graphics, haptics, and collision-sensing physical modeling, we can thus enhance our 4D visualization experience. Since we cannot actually place interaction devices in 4D, we develop fluid methods for interacting with a 4D object in its 3D shadow image using adapted reduced-dimension 3D tools for manipulating objects embedded in 4D. By physically modeling the correct properties of 4D surfaces, their bending forces, and their collisions in the 3D interactive or haptic controller interface, we can support full-featured physical exploration of 4D mathematical objects in a manner that is otherwise far beyond the real-world experience accessible to human beings

Design and Simulation of a Mechanical Hand

A variety of mechanical hand designs have been developed in the past few decades. The majority of the designs were made with the sole purpose of imitating the human hand and its capabilities; however, none of these designs have been equipped with all the motions and sensory capabilities of the human hand. The primary goal of this thesis project was to design a robotic hand with the required amount of degrees-of-freedom and necessary constraints to achieve all the motions of the human hand. Demonstration of the American Sign Language (ASL) alphabet, using a virtual design and controls platform, was used as a means of proving the dexterity of the designed hand. The objectives of the thesis were accomplished using a combination of computerized 3-D modeling, kinematic modeling, and LabView programming. A mechanical hand model was designed using SolidWorks. Actuation methods were incorporated into the design based on the structure of the connecting tendons in the human hand. To analyze the motions of the mechanical hand model, finger assemblies were manufactured at two different scales (full and ¼ size) using rapid prototyping. These finger assemblies were used to study the developed forces within the joints prone to failure when subjected to actuation and spring forces. A free body diagram and an Ansys model were created to quantify the force and stress concentrations at the contact point of the pin joint in the distal interphalangeal joint, a location of failure in the rapid prototype assembly. A complete kinematic model was then developed for the mechanical hand using the Denavit-Hartenberg principle to map all the joints of the hand and finger tips in a universal frame of reference. A program was developed using LabView and Matlab software tools to incorporate the developed kinematic model of the designed hand and plot the 3-D locations of all joints in the universal frame of reference for each letter of the ASL alphabet. The program was then interfaced with the SolidWorks hand assembly to virtually control the motions of the designed assembly and to optimize the hand motions. In summary, a mechanical human hand model and interacting software platform were developed to simulate the dexterity of a designed human hand and to implement virtual controls, based on kinematic modeling, to achieve the optimum motion patterns needed to demonstrate the ASL alphabet. The designed hand was capable of performing all the static gestures of the ASL alphabet

Facial Modelling and animation trends in the new millennium : a survey

M.Sc (Computer Science)Facial modelling and animation is considered one of the most challenging areas in the animation world. Since Parke and Waters’s (1996) comprehensive book, no major work encompassing the entire field of facial animation has been published. This thesis covers Parke and Waters’s work, while also providing a survey of the developments in the field since 1996. The thesis describes, analyses, and compares (where applicable) the existing techniques and practices used to produce the facial animation. Where applicable, the related techniques are grouped in the same chapter and described in a chronological fashion, outlining their differences, as well as their advantages and disadvantages. The thesis is concluded by exploratory work towards a talking head for Northern Sotho. Facial animation and lip synchronisation of a fragment of Northern Sotho is done by using software tools primarily designed for English.Computin

基于点缓存全局光照技术的研究

As the development of animation industry, there are more and more requirements for the realistic rendering. Global illumination (GI) is an significant part of realistic rendering, and it has been focused by researchers for many years. Not only the lighting directly from the light sources but also the lighting reflected by other objects in the scene is required to be computed, so it's complicated to solve the GI problem that can be described as the rendering equation. Several algorithms can be used to solve this problem, such as Monte Carlo based ray tracing, photon mapping, many lights based approaches, point based global illumination and so on. This thesis is about Point Based Global Illumination (PBGI). PBGI is a popular rendering algorithm in movie and motion picture productions. This algorithm provides a diffuse global illumination solution by caching radiance in a mesh-less hierarchical data structure during a pre-process, while solving for the visibility over this cache, at rendering time, for each receiver, using a microbuffer, which is a localized depth and color buffer inspired from real time rendering environments. As a result, noise free ambient occlusion, indirect soft shadows and color bleeding effects are computed efficiently for high resolution image output and in a temporally coherent fashion. PBGI has attracted increasing attention nowadays because of its efficiency and noise free quality. However, there are still some problems, such as it can not simulation non-diffuse light transport, that makes it have limited applications. My thesis aims to solve these issues in PBGI and extend it to support more light transport path. Based on the spatial coherency, we propose a factorized solution of PBGI to make it more efficient by reusing the tree cut and the microbuffers. In PBGI, each receiver traverse the point cloud tree independently, but we observe that the similar receivers have the similar tree cut, that means there is redundancy during the traversal process. A similar model of the receivers is proposed at first, and then it is used to cluster the receivers. The point cloud tree is traversed for a cluster instead of for each receiver, and a cluster tree cut is obtained. The far nodes in the cluster tree cut are shared directly by all the receivers in this cluster without further traversing, while the near nodes are traversed for each receiver independently. A cluster microbuffer is proposed to solve for the visibility of the far nodes, while the receiver specific microbuffer is used to solve for the visibility of the near nodes and the refined nodes. The final microbuffer by combining these two microbuffers is convolved with the bidirectional reflection distribution funcion (BRDF) of the receiver to get the final indirect illumination. Our algorithm offers a significant rendering speed-up for a negligible and controllable approximation error and it inherits the temporal coherence of PBGI. We also propose a wavelet based solution to PBGI to compute the non-diffuse light transport.As only the diffuse lighting of the point is baked in PBGI, it can not simulate the non-diffuse light transport, such as caustics. PBGI tree nodes uses spherical harmonics (SH) to represent outgoing radiance. Unfortunately, even using a larger number of coefficients, SH are not able to capture high frequencies efficiently, which translates in our case to non-diffuse reflections or refractions. Consequently, caustics stemming from metals, plastics, glass and other reflective or refractive materials are not handled with classical PBGI frameworks. Even when ignoring the performance issue induced by a larger number of SH coefficients, ringing artifacts quickly appear. Compared with SH, haar wavelets support non-linear approximation, so the the representation is compact. So we propose to represent the outgoing radiance of the non-diffuse point with wavelet coefficients by sampling according to a cube map firstly and wavelet transforming each face of this cube map. The coefficients are further encoded hierarchically in the point cloud tree to decrease the memory usage, that means the coefficients themselves are wavelet transformed, generating two kinds of coefficients: node approximation coefficients and node detail coefficients. The node approximation coefficients are stored for the low level nodes (close to the root), and the node detail coefficients are stored for the high level nodes. To avoid storing the entire list of nodes vectors at any intermediate state, we compute this compressed representation during a post-order depth-first traversal of the PBGI tree. Further more, according to the artifacts problem that appears when there is high frequency BRDF or lighting in the scene, we propose to use the importance driven microbuffer. The importance function that includes the incoming lighting importance and the BRDF importance is used to drive the microbuffer, that means when one pixel has high frequency information (lighting or BRDF), it will be subdivided. Finally, our rendering algorithm allows to handle non-diffuse light transport, reproducing caustics with a similar quality to bidirectional path tracing for only a fraction of the computation time, with an intuitive control on the approximation error. Based on the previous two algorithms, we propose a view-tree based approach to compute the multiple bounces reflection. In PBGI, the indirect illumination of each point in the point cloud needs to be evaluated by traversing the point cloud tree and splatting the nodes in the tree-cut, so each point is treated a receiver. We propose to organize all the receivers into a view tree, and the point cloud tree is traversed for the view tree instead of for each receiver. The view tree approach is based on the observation some nodes in the point cloud tree contribute similarly to all the points in other nodes, that means we don¡¯t need to traverse for these points respectively but only for the node that contains these points. This is an extension to the factorized PGBI from one level (cluster) to a hierarchical structure (tree). Another problem for multiple bounces computation is how to evaluate the outgoing radiance from the incoming radiance efficiently. As the outgoing radiance needs to be computed for each sampling direction, the time complexity is $O(n^4)$, where $n\times n$ represents the resolution of the hemisphere or square according to which the incoming direction and the outgoing direction are sampled. The wavelet representation is sparse that improves the performance, so we decide to wavelet transform the incoming radiance and the BRDF and multiply them in the frequency domain. We propose a novel outgoing radiance computation model by doing product between 4D BRDF wavelet coefficients and 2D incoming radiance wavelet coefficients.Finally, the point cloud tree with multiple bounces reflection stored is used to offer a preview rendering of the scene by utilizing the GPU computing efficiently. We can support scenes that include diffuse materials and all frequency glossy materials with a changing camera. My thesis improves and extends the PGBI algorithm so that it can be used in more applications.Comme le développement de l'industrie de l'animation, il ya de plus en plus d'exigences pour le rendu réaliste. L'illumination globale (IG) est une partie importante de rendu réaliste, et il a été porté par les chercheurs depuis de nombreuses années. Non seulement la lumière directement à partir des sources de lumière, mais aussi l'éclairage réfléchi par les autres objets de la scène doit être calculé, de sorte qu'il est compliqué de résoudre le problème de l'IG qui peut être décrit par l'équation de rendu. Plusieurs algorithmes peuvent être utilisés pour résoudre ce problème, comme le traçage basé Monte Carlo ray, le photon mapping, beaucoup de lumières approches fondées, le point sur la base de l'illumination globale et ainsi de suite. Cette thèse est basé sur des points sur Global Illumination (PBGI). PBGI est un algorithme de rendu populaire en cinéma et productions cinématographiques. Cet algorithme fournit une solution d'illumination globale diffuse en mettant en cache l'éclat dans une structure de données hiérarchique maille moins lors d'un pré-traitement, tout en résolvant pour la visibilité sur ce cache, au moment du rendu, pour chaque récepteur, en utilisant un microbuffer, qui est un localisée profondeur et tampon de couleurs inspirée des environnements de rendu en temps réel. En conséquence, le bruit ambiant sans occlusion, ombres douces indirects et des saignements de couleur effets sont calculés de manière efficace pour la sortie image haute résolution et d'une manière temporellement cohérente. PBGI a attiré une attention croissante de nos jours, en raison de sa qualité, sans l'efficacité et le bruit. Cependant, il ya encore quelques problèmes, comme il ne peut pas la simulation de transport de lumière non diffuse, ce fait avoir des applications limitées. Ma thèse vise à résoudre ces questions dans PBGI et l'étendre à soutenir chemin de transport de plus de lumière.Basé sur la cohérence spatiale, nous proposons une solution factorisée de PBGI à faire plus efficace en réutilisant la coupe des arbres et les microbuffers. Dans PBGI, chaque récepteur parcourir l'arborescence point de trouble indépendamment, mais nous observons que les récepteurs semblables ont la coupe d'arbre similaire, cela signifie qu'il ya une redondance au cours du processus de traversée. Un modèle similaire de récepteurs est proposé dans un premier temps, puis il est utilisé pour regrouper les récepteurs. L'arbre de nuage de points est traversé d'un cluster au lieu de pour chaque récepteur, et un arbre coupé de cluster est obtenu. Les nœuds loin dans l'arbre coupé du cluster sont partagés directement par tous les récepteurs de ce groupe sans autre déplacement, tandis que les nœuds proches sont traversés pour chaque récepteur indépendamment. Un microbuffer de cluster est proposé à résoudre pour la visibilité des nœuds loin, tandis que le microbuffer spécifique récepteur est utilisé à résoudre pour la visibilité des près de noeuds et les noeuds raffinés. Le microbuffer finale en combinant ces deux microbuffers est convoluée avec le funcion de distribution de réflexion bidirectionnelle (BRDF) du récepteur pour obtenir l'éclairage indirect finale. Notre algorithme offre un rendu accélération significative pour une erreur d'approximation négligeable et contrôlable et il hérite de la cohérence temporelle de PBGI.Nous proposons également une solution d'ondelettes basée à PBGI pour calculer les transport.As lumineuses non diffuses que l'éclairage diffus du point est cuit dans PBGI, il ne peut pas simuler la non-diffus transport léger, comme les caustiques. Nœuds d'arbre PBGI utilise des harmoniques sphériques (SH) pour représenter le rayonnement sortant. Malheureusement, même en utilisant un plus grand nombre de coefficients, SH ne sont pas en mesure de capturer efficacement les hautes fréquences, ce qui se traduit dans notre cas à des réflexions ou réfractions non-diffuses. Par conséquent, caustiques découlant de métaux, les plastiques, le verre et d'autres matériaux réfléchissants ou de réfraction ne sont pas traitées avec des cadres de PBGI classiques. Même lorsque ignorant la question de la performance induite par un plus grand nombre de coefficients SH, sonner artefacts apparaissent rapidement. Par rapport aux SH, les ondelettes de Haar supportent approximation non linéaire, de sorte que la représentation est compact. Donc, nous proposons de représenter le rayonnement sortant du point de non-diffuse avec des coefficients d'ondelettes par échantillonnage selon un cube map abord et ondelettes transformer chaque face de ce cube map. Les coefficients sont en outre codées hiérarchiquement dans l'arbre de nuages ​​de points pour réduire l'utilisation de mémoire, cela signifie que les coefficients sont eux-mêmes transformés en ondelettes, générer deux types de coefficients: noeud coefficients d'approximation et coefficients de détail de noeud. Les coefficients d'approximation noeud sont stockées pour les nœuds de bas niveau (à proximité de la racine), et les coefficients de détail de noeud sont stockées pour les nœuds de haut niveau. Pour éviter de stocker la liste complète des noeuds vecteurs à tout état intermédiaire, nous calculons cette représentation comprimé lors d'un post-order profondeur d'abord la traversée de l'arbre de PBGI. De plus, selon le problème des artefacts qui apparaît quand il ya BRDF haute fréquence ou de l'éclairage de la scène, nous proposons d'utiliser l'importance entraîné microbuffer. La fonction d'importance qui comprend l'importance de l'éclairage entrant et l'importance BRDF est utilisé pour entraîner le microbuffer, cela signifie quand un pixel dispose d'informations à haute fréquence (éclairage ou BRDF), elle sera découpée. Enfin, notre algorithme de rendu permet de gérer le transport de la lumière non diffuse, la reproduction caustiques avec une qualité similaire à la trajectoire bidirectionnelle traçage pour seulement une fraction du temps de calcul, avec un contrôle intuitif sur l'erreur d'approximation.Basé sur les deux algorithmes précédents, nous proposons une approche basée vue arbre pour calculer la rebonds réflexion multiple. Dans PBGI, l'éclairage indirect de chaque point dans le nuage de points doit être évaluée en parcourant l'arborescence point de trouble et splatting les nœuds de l'arbre-coupe, de sorte que chaque point est traité un récepteur. Nous proposons d'organiser tous les récepteurs dans un arbre de la vue, et l'arbre de nuage de points est traversée pour l'arbre de vue au lieu de pour chaque récepteur. L'approche de l'arbre de la vue est basée sur l'observation des noeuds de l'arbre de nuage de points contribuent de manière similaire à tous les points dans d'autres nœuds, cela signifie que nous ¡¯ t besoin de traverser pour ces points, respectivement, mais seulement pour le nœud qui contient ces points. Ceci est une extension de la PGBI factorisée d'un niveau (cluster) à une structure hiérarchique (arbre). Un autre problème pour de multiples rebonds calcul est comment évaluer le rayonnement sortant du rayonnement entrant efficacement. Comme le rayonnement sortant doit être calculé pour chaque direction d'échantillonnage, la complexité est en O $(n ^ 4)$, où n $\ times n$ représente la résolution de l'hémisphère ou carré selon laquelle le sens entrant et sortant direction sont échantillonnés. La représentation en ondelettes est clairsemée qui améliore la performance, donc nous avons décidé de la transformée en ondelettes le rayonnement entrant et la BRDF et les multiplier dans le domaine de fréquence. Nous proposons un modèle roman de calcul de rayonnement sortant en faisant produit entre 4D BRDF coefficients d'ondelettes et 2D radiance entrant coefficients d'ondelettes.Enfin, l'arbre de nuage de points avec de multiples rebonds réflexion stockée est utilisée pour offrir un rendu d'aperçu de la scène en utilisant efficacement le GPU computing. Nous pouvons soutenir scènes qui comprennent des matériaux diffuses et toutes les fréquences matériaux brillants avec une caméra changer. Ma thèse améliore et prolonge la PGBI algorithme de sorte qu'il peut être utilisé dans plusieurs applications

Simulación visual de la iluminación : teoría, técnicas, análisis de casos

Descripció del recurs: 10 de desembre de 2015La simulación de la iluminación es uno de los grandes temas implicados en la creación de escenarios virtuales. Requiere asimilar diversas técnicas que derivan de principios teóricos, sin cuyo conocimiento será difícil utilizar bien estas técnicas y también se necesita una buena formación visual, la capacidad de observar, de razonar visualmente, de reflexionar sobre los causas que subyacen a los fenómenos visuales, a las apariencias de objetos y escenarios familares. La finalidad de este libro, que se publica en paralelo con otro sobre simulación visual de materiales, es abordar todos estos temas en profundidad, de un modo coherente. En su Iª parte, ofrece una visión general de los fundamentos teóricos de las técnicas de simulación visual, tanto por lo que hace a los conceptos y recursos disponibles, como a las capacidades y límitaciones de nuestro sistema visual. En su IIº parte, se desarrollan ejemplos de aplicación que faciliten su asimilación y su utilización práctica