Multi-touch 3D Exploratory Analysis of Ocean Flow Models

Modern ocean flow simulations are generating increasingly complex, multi-layer 3D ocean flow models. However, most researchers are still using traditional 2D visualizations to visualize these models one slice at a time. Properly designed 3D visualization tools can be highly effective for revealing the complex, dynamic flow patterns and structures present in these models. However, the transition from visualizing ocean flow patterns in 2D to 3D presents many challenges, including occlusion and depth ambiguity. Further complications arise from the interaction methods required to navigate, explore, and interact with these 3D datasets. We present a system that employs a combination of stereoscopic rendering, to best reveal and illustrate 3D structures and patterns, and multi-touch interaction, to allow for natural and efficient navigation and manipulation within the 3D environment. Exploratory visual analysis is facilitated through the use of a highly-interactive toolset which leverages a smart particle system. Multi-touch gestures allow users to quickly position dye emitting tools within the 3D model. Finally, we illustrate the potential applications of our system through examples of real world significance

Real-Time Markerless Tracking the Human Hands for 3D Interaction

This thesis presents methods for enabling suitable human computer interaction using only movements of the bare human hands in free space. This kind of interaction is natural and intuitive, particularly because actions familiar to our everyday life can be reflected. Furthermore, the input is contact-free which is of great advantage e.g. in medical applications due to hygiene factors. For enabling the translation of hand movements to control signals an automatic method for tracking the pose and/or posture of the hand is needed. In this context the simultaneous recognition of both hands is desirable to allow for more natural input. The first contribution of this thesis is a novel video-based method for real-time detection of the positions and orientations of both bare human hands in four different predefined postures, respectively. Based on such a system novel interaction interfaces can be developed. However, the design of such interfaces is a non-trivial task. Additionally, the development of novel interaction techniques is often mandatory in order to enable the design of efficient and easily operable interfaces. To this end, several novel interaction techniques are presented and investigated in this thesis, which solve existing problems and substantially improve the applicability of such a new device. These techniques are not restricted to this input instrument and can also be employed to improve the handling of other interaction devices. Finally, several new interaction interfaces are described and analyzed to demonstrate possible applications in specific interaction scenarios.Markerlose Verfolgung der menschlichen Hände in Echtzeit für 3D Interaktion In der vorliegenden Arbeit werden Verfahren dargestellt, die sinnvolle Mensch- Maschine-Interaktionen nur durch Bewegungen der bloßen Hände in freiem Raum ermöglichen. Solche "natürlichen" Interaktionen haben den besonderen Vorteil, dass alltägliche und vertraute Handlungen in die virtuelle Umgebung übertragen werden können. Außerdem werden auf diese Art berührungslose Eingaben ermöglicht, nützlich z.B. wegen hygienischer Aspekte im medizinischen Bereich. Um Handbewegungen in Steuersignale umsetzen zu können, ist zunächst ein automatisches Verfahren zur Erkennung der Lage und/oder der Art der mit der Hand gebildeten Geste notwendig. Dabei ist die gleichzeitige Erfassung beider Hände wünschenswert, um die Eingaben möglichst natürlich gestalten zu können. Der erste Beitrag dieser Arbeit besteht aus einer neuen videobasierten Methode zur unmittelbaren Erkennung der Positionen und Orientierungen beider Hände in jeweils vier verschiedenen, vordefinierten Gesten. Basierend auf einem solchen Verfahren können neuartige Interaktionsschnittstellen entwickelt werden. Allerdings ist die Ausgestaltung solcher Schnittstellen keinesfalls trivial. Im Gegenteil ist bei einer neuen Art der Interaktion meist sogar die Entwicklung neuer Interaktionstechniken erforderlich, damit überhaupt effiziente und gut bedienbare Schnittstellen konzipiert werden können. Aus diesem Grund wurden in dieser Arbeit einige neue Interaktionstechniken entwickelt und untersucht, die vorhandene Probleme beheben und die Anwendbarkeit eines solchen Eingabeinstruments für bestimmte Arten der Interaktion verbessern oder überhaupt erst ermöglichen. Diese Techniken sind nicht auf dieses Eingabeinstrument beschränkt und können durchaus auch die Handhabung anderer Eingabegeräte verbessern. Des Weiteren werden mehrere neue Interaktionsschnittstellen präsentiert, die den möglichen Einsatz bloßhändiger Interaktion in verschiedenen, typischen Anwendungsgebieten veranschaulichen

Molecular manipulation in augmented reality: A user experience design applied research on new paradigms of interaction.

This thesis presents the further developments of VR CHEM (2017), a molecular manipulation software in Virtual Reality. After evaluating a shift to Augmented Reality, changing consequently the name to AR CHEM, the project focused on this technology as it offers a competitive advantage for future development, especially considering the less invasive user experience and the incoming investments in the market. The project is developed using HoloLens and Unity 3D as a platform, as these tools are considered the state of the art for AR technologies (2018). The thesis presents a first iteration of new paradigms of interactions in AR: the 3D user interfaces and the Platform System. While the first explore the possibility of giving information to the user through 3D models used as UI/UX indications, the second strengthen the user’s visual perception of depth in space, tying all the content to more tangible elements

NASA space station automation: AI-based technology review

Research and Development projects in automation for the Space Station are discussed. Artificial Intelligence (AI) based automation technologies are planned to enhance crew safety through reduced need for EVA, increase crew productivity through the reduction of routine operations, increase space station autonomy, and augment space station capability through the use of teleoperation and robotics. AI technology will also be developed for the servicing of satellites at the Space Station, system monitoring and diagnosis, space manufacturing, and the assembly of large space structures

Photogrammetry as a surveying thechnique apllied to heritage constructions recording - avantages and limitations

Dissertação de Mestrado Integrado em Arquitetura, com a especialização em Arquitetura apresentada na Faculdade de Arquitetura da Universidade de Lisboa para obtenção do grau de Mestre.A presente dissertação tem por objectivo investigar e evidenciar as vantagens da aplicação da fotogrametria, e possíveis integrações com outros métodos de levantamento, como seja o varrimento laser terrestre, posicionamento por GPS, entre outros, para realizar levantamentos de construções patrimoniais ou eruditas e a respectiva produção de documentação base para viabilizar intervenções de conservação, restauro ou reabilitação. A motivação para a investigação advém da aplicação flexível, versátil, simples, acessível, e baixo-custo da fotogrametria em projectos de levantamento pequenos ou extensos. Tenciona-se igualmente colmatar as desvantagens tradicionais da fotogrametria, nomeadamente a transição entre espaços interiores e exteriores, e registo de espaços estreitos, de difícil acesso, e de geometrias complexas, num único projecto de documentação. Pretende-se ultrapassar estas dificuldades através da utilização máxima das potencialidades da fotogrametria com o uso de imagens olho de peixe e apenas como último recurso utilizar instrumentos complementares. No caso de estudo principal, o Castelo do Convento de Cristo, demonstra-se a aplicação dos métodos investigados. Nos casos de estudo secundários abordam-se problemas parcelares, desde elementos decorativos até à totalidade do edificado: Convento dos Capuchos, em Sintra; Alcáçova e trecho de muralha do Castelo de Sesimbra; Igreja de Stº André, em Mafra; entre outros. Os casos auxiliaram na determinação de procedimentos a generalizar posteriormente. Por fim, propõem-se algoritmos que auxiliam na produção de documentação.ABSTRACT: The present dissertation aims to research and demonstrate the advantages of the application of photogrammetry, and its possible integrations with other methods, such as terrestrial laser scanning, GPS positioning, and among others, to perform surveys of heritage or erudite buildings and respective production of base documentation to enable interventions of conservation, restoration, or rehabilitation. The motivation for researching is due to the flexible, versatile, simple, affordable, and low-cost application of photogrammetry in small and extensive survey projects. It is also intended to overcome the traditional disadvantages of photogrammetry, such as the transition between interior and exterior spaces, and difficulty of recording narrow, hard-to-access, and complex geometric spaces, in a single project. It is intended to overcome such challenges by maximizing the potential uses of photogrammetry with the use of fisheye images and by using other survey instruments as a last resort. In the main case study, the Castle of the Convent of Christ, the application of the investigated methods is demonstrated. In the secondary case studies, partial problems are addressed, ranging from decorative elements to the entire building: Convento dos Capuchos, in Sintra; Citadel and section of a wall of the Castle of Sesimbra; Igreja de St André, in Mafra; among others; The case studies aided in determining general procedures. Finally, algorithms that accelerate the production of documentation are proposed.N/

Assessment of Structure from Motion for Reconnaissance Augmentation and Bandwidth Usage Reduction

Modern militaries rely upon remote image sensors for real-time intelligence. A typical remote system consists of an unmanned aerial vehicle, or UAV, with an attached camera. A video stream is sent from the UAV, through a bandwidth-constrained satellite connection, to an intelligence processing unit. In this research, an upgrade to this method of collection is proposed. A set of synthetic images of a scene captured by a UAV in a virtual environment is sent to a pipeline of computer vision algorithms, collectively known as Structure from Motion. The output of Structure from Motion, a three-dimensional model, is then assessed in a 3D virtual world as a possible replacement for the images from which it was created. This study shows Structure from Motion results from a modifiable spiral flight path and compares the geoaccuracy of each result. A flattening of height is observed, and an automated compensation for this flattening is performed. Each reconstruction is also compressed, and the size of the compression is compared with the compressed size of the images from which it was created. A reduction of 49-60% of required space is shown