1,384 research outputs found

    Sketching-out virtual humans: From 2d storyboarding to immediate 3d character animation

    Get PDF
    Virtual beings are playing a remarkable role in today’s public entertainment, while ordinary users are still treated as audiences due to the lack of appropriate expertise, equipment, and computer skills. In this paper, we present a fast and intuitive storyboarding interface, which enables users to sketch-out 3D virtual humans, 2D/3D animations, and character intercommunication. We devised an intuitive “stick figurefleshing-outskin mapping” graphical animation pipeline, which realises the whole process of key framing, 3D pose reconstruction, virtual human modelling, motion path/timing control, and the final animation synthesis by almost pure 2D sketching. A “creative model-based method” is developed, which emulates a human perception process, to generate the 3D human bodies of variational sizes, shapes, and fat distributions. Meanwhile, our current system also supports the sketch-based crowd animation and the storyboarding of the 3D multiple character intercommunication. This system has been formally tested by various users on Tablet PC. After minimal training, even a beginner can create vivid virtual humans and animate them within minutes

    Sketching-out virtual humans: A smart interface for human modelling and animation

    Get PDF
    In this paper, we present a fast and intuitive interface for sketching out 3D virtual humans and animation. The user draws stick figure key frames first and chooses one for “fleshing-out” with freehand body contours. The system automatically constructs a plausible 3D skin surface from the rendered figure, and maps it onto the posed stick figures to produce the 3D character animation. A “creative model-based method” is developed, which performs a human perception process to generate 3D human bodies of various body sizes, shapes and fat distributions. In this approach, an anatomical 3D generic model has been created with three distinct layers: skeleton, fat tissue, and skin. It can be transformed sequentially through rigid morphing, fatness morphing, and surface fitting to match the original 2D sketch. An auto-beautification function is also offered to regularise the 3D asymmetrical bodies from users’ imperfect figure sketches. Our current system delivers character animation in various forms, including articulated figure animation, 3D mesh model animation, 2D contour figure animation, and even 2D NPR animation with personalised drawing styles. The system has been formally tested by various users on Tablet PC. After minimal training, even a beginner can create vivid virtual humans and animate them within minutes

    Interaction in virtual environments with the upper body

    Get PDF
    Tese de mestrado em Engenharia Informática, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2012Pode-se considerar que a Realidade Virtual é uma interface entre o utilizador e um sistema computacional, cujo objetivo principal é simular um Envolvimento Virtual realístico no qual é possível navegar e com o qual se pode interagir em tempo real. O objetivo é proporcionar ao utilizador uma forte sensação de imersão no Envolvimento Virtual ou seja, a sensação de presença física e efetiva nesse envolvimento. Para haver o máximo de imersividade nos Envolvimentos Virtuais são usados diversos dispositivos para que a navegação e interação sejam o mais credíveis possível, dispositivos tais como Head-Mounted Displays, luvas de dados, de rastreamento e dispositivos que geram sensações de tato e força (feedback háptico). Alguns sistemas dispõem de superfícies de representação de grandes dimensões como por exemplo a CAVE. Atualmente a Realidade Virtual é utilizada em diversas áreas porque é uma forma de simular uma experiência próxima da realidade reduzindo, em alguns casos, o perigo que existe no mundo real e permitindo de forma fácil a repetição de situações ou experiências. A Realidade Virtual é aplicada em áreas tão diversas como a Medicina para treino cirúrgico em pacientes virtuais, o entretenimento com os jogos e filmes tridimensionais, a Psicologia no tratamento de fobias e traumas, entre outras. Este projeto, intitulado “Interaction in Virtual Environments with the Upper Body”, desenvolveu-se no âmbito da Realidade Virtual e enquadrou-se no projeto “Future Safety Warnings: Virtual Reality in the study of technology-based warnings” financiado pela Fundação para a Ciência e a Tecnologia (PTDC/PSI-PCO/100148/2008) que se contextualiza na área de Ergonomia. Este trabalho foi realizado no Laboratório de Ergonomia (ErgoLAB) da Faculdade de Motricidade Humana da Universidade Técnica de Lisboa, mais especificamente na unidade de Realidade Virtual chamada ErgoVR, e contou com o trabalho de uma equipa multidisciplinar composta por Ergonomistas, Psicólogos, Engenheiros Informáticos, Arquitetos, Designers entre outros. Para suportar todos os projetos realizados no ErgoVR existe um sistema de Realidade Virtual com o mesmo nome. No projeto “Future Safety Warnings: Virtual Reality in the study of technology-based warnings” a Realidade Virtual é utilizada para avaliar a consonância comportamental dos participantes perante avisos de segurança em situações de emergência no interior de edifícios. O projeto descrito neste documento veio resolver dois aspetos que podem afetar a imersão: (1) o facto de o utilizador não ser representado por nenhum Humano Virtual; e (2) o facto de a interação com os objetos do Ambiente Virtual ser limitada e pouco natural. Existe um sistema de interação que funciona da seguinte forma: é colocado um sensor na mão do participante, quando este se encontra num mínimo de uma distância pré-definida do botão, bastava esticar a mão para o sistema detetar que tinha havido um movimento e desencadear o evento associado à ação. No entanto este sistema tem algumas limitações visto que o único feedback visual que o participante tinha era um cursor bidimensional o que limita a perceção de distância do objeto que o participante quererá interagir. Para além desta limitação, o sistema apenas permitia a ação de pressionar. Assim, o objetivo principal deste projeto é: (1) a criação de um Humano Virtual; (2) a possibilidade de reproduzir os movimentos do participante e refleti-los no Humano Virtual; e (3) permitir que o sistema suporte mais ações como por exemplo agarrar ou largar. Com a criação do Humano Virtual e a reflexão dos movimentos do participante, existirá uma maior perceção de distância no Envolvimento Virtual. Para cumprir com este objetivo utilizaram-se sensores de movimento que captam a orientação 3D e dados cinemáticos relativos aos membros superiores. Estes sensores de movimento são colocados no braço, antebraço e mão, para capturar os movimentos e orientações reais do participante e passá-los para os membros do Humano Virtual, cujo modelo foi criado previamente. A este modelo e a todos os elementos do Envolvimento Virtual associaram-se características físicas, como por exemplo a massa, para dar realismo e credibilidade à simulação e fazer com que passasse a ser possível a interação do utilizador com determinados objetos presentes no ambiente. O projeto descrito nesta tese envolveu quatro etapas. A primeira etapa foi de familiarização com o sistema ErgoVR, com as ferramentas de desenvolvimento nele utilizadas e foi realizado um levantamento do estado da arte. Nesta etapa também se criou um Humano Virtual com as ferramentas de modelação 3D que permitiram criar um modelo com esqueleto, animações e texturas e que tornaram possível exportá-lo para o formato utilizado no sistema ErgoVR. Na segunda etapa tratou-se de todas as questões relativas aos sensores, leitura de dados, transformações dos ângulos de Euler e transposição dos dados provenientes dos sensores no Humano Virtual. A terceira etapa foi relativa à simulação das regras da física Newtoniana dentro do Envolvimento Virtual. A quarta etapa foi relativa às formas de interação do utilizador com os objetos do Envolvimento Virtual, como agarrar, largar, puxar e pressionar. Na etapa de familiarização decidiu-se que este projeto iria desenvolver-se sobre a plataforma Microsoft .NET (DotNET) visto o sistema ErgoVR ter sido desenvolvido sobre essa mesma plataforma. Deste modo garantia-se a integração do resultado deste projeto no sistema ErgoVR de uma forma mais simplificada. Ao mesmo tempo, as características da plataforma .NET enquadravam-se com as necessidades deste projeto. Na fase inicial do projeto houve o envolvimento num trabalho de equipa que foi fundamental para a etapa inicial de familiarização e para compreender o funcionamento do sistema ErgoVR. Deste envolvimento surgiu a colaboração num artigo científico, elaborado pela equipa do laboratório, como co-autora, intitulado “Using space exploration matrices to evaluate interaction with Virtual Environments”. Neste artigo descreve-se um estudo realizado com o sistema ErgoVR que avalia as decisões dos utilizadores perante a influência da informação de segurança colocada nos ambientes quando são confrontados com situações de emergência. São recolhidos durante a simulação, entre outros, dados relativos à posição do participante, às distâncias percorridas, aos tempos do percurso. Com estes dados são geradas matrizes a partir das quais é possível identificar especificamente os fluxos e as zonas do ambiente que são mais visitadas pelos participantes, e deste modo avaliar o seu comportamento. Na modelação do Humano Virtual, ainda na fase inicial do projeto, foram realizados testes exaustivos utilizando diversas ferramentas de modelação com o intuito de identificar as ferramentas que exportam o modelo para um formato aceite pelo sistema ErgoVR mantendo correta toda a informação necessária associada (malha, texturas, cabelo, animações corporais e faciais e roupas). Concluiu-se que a melhor abordagem seria trabalhar com o 3DS Max e com o Daz 3D. Na segunda etapa foi implementada a biblioteca responsável por fazer a leitura dos dados dos sensores e transpô-los para os ossos do Humano Virtual. Esta biblioteca é composta por quatro classes, cada uma com funções diferentes. No final desta fase já era possível colocar os sensores no utilizador e ver o Humano Virtual refletir os movimentos no ambiente. Na terceira etapa, para realizar a biblioteca de física, foi necessário levar a cabo um levantamento sobre questões de física. Para não haver perda de imersão do utilizador no ambiente é necessário impedir situações como: (1) o Humano Virtual trespassar as paredes; ou (2) o Humano Virtual levar a mão a um objeto, e esta atravessá-lo. Para tal, todo o objeto tem de ter associadas massa, gravidade e forças para se deslocar no ambiente. Na última etapa, para desenvolver a biblioteca de interação com objetos, foi necessário fazer um levantamento sobre quais as ações que um utilizador podia realizar no sistema. Conclui-se que seria necessário poder agarrar, largar, puxar e pressionar. Ao realizar esta biblioteca decidiu-se que alguns objetos virtuais tinham de ter associados a si informação extra sobre os possíveis modos de interação que o utilizador pode realizar sobre eles. As bibliotecas desenvolvidas neste projeto constituem um módulo perfeitamente integrável no sistema ErgoVR e permitem a utilização de sensores nos membros superiores do utilizador cujos movimentos são refletidos no Humano Virtual correspondente. O utilizador pode, de modo natural, interagir com elementos do ambiente realizando gestos relativos às ações de agarrar, largar, puxar e pressionar. Considerou-se que o módulo desenvolvido é uma mais-valia para o ErgoVR porque permite a aplicação deste sistema de Realidade Virtual a diferentes cenários em diversos âmbitos, sempre que a interação de um utilizador humano com objectos presentes no Envolvimento Virtual seja requerida.The Virtual Reality (VR) is an interface between the user and a system and is main goal is simulate a Virtual Environment (VE) next to the reality. The advantage of use VR is the possibility of simulates the dangerous that exist in real world or allowing the repetition of situations and experiences. Nowadays VR is used in many areas, from Medicine, for surgical training in virtual patients, to the army in which the soldiers do virtual training. The project “Interaction in Virtual Environments with the Upper Body” is developed in the context of Virtual Reality and is a part of the project “Future Safety Warnings: Virtual Reality in the study of technology-based warnings” funded by the Portuguese Science Foundation (PTDC/PSI-PCO/100148/2008) . This project was developed in the Ergonomics Laboratory (ErgoLAB) of the Faculty of Human Kinetics of the Technical University of Lisbon more specifically in the research unit ErgoVR. In the project “Future Safety Warnings: Virtual Reality in the study of technology-based warnings”, VR is used to evaluate the participant’s behavior towards safety warnings in emergencies inside buildings. In this project the interaction was weak and due to the fact of a participant do not have virtual representation, the immersion was less. Therefore, the main goal of the described project is give to the participant the possibility of see is upper limbs’ movements reflected in a Virtual Human (VH) inside the VE, and have the possibility to interact with virtual objects, give more sensation of immersion. To complete with this goal sensors were used and place in the arm, to capture the participant’s movements. A VH was created to performed the participant´s movements in the VE. To this VH and to the VE, physics elements were added to give more credibility to the simulation and make possible the interaction with objects in the scene

    RGB-D-based Action Recognition Datasets: A Survey

    Get PDF
    Human action recognition from RGB-D (Red, Green, Blue and Depth) data has attracted increasing attention since the first work reported in 2010. Over this period, many benchmark datasets have been created to facilitate the development and evaluation of new algorithms. This raises the question of which dataset to select and how to use it in providing a fair and objective comparative evaluation against state-of-the-art methods. To address this issue, this paper provides a comprehensive review of the most commonly used action recognition related RGB-D video datasets, including 27 single-view datasets, 10 multi-view datasets, and 7 multi-person datasets. The detailed information and analysis of these datasets is a useful resource in guiding insightful selection of datasets for future research. In addition, the issues with current algorithm evaluation vis-\'{a}-vis limitations of the available datasets and evaluation protocols are also highlighted; resulting in a number of recommendations for collection of new datasets and use of evaluation protocols

    Motion Capture Dataset for Practical Use of AI-based Motion Editing and Stylization

    Full text link
    In this work, we proposed a new style-diverse dataset for the domain of motion style transfer. The motion dataset uses an industrial-standard human bone structure and thus is industry-ready to be plugged into 3D characters for many projects. We claim the challenges in motion style transfer and encourage future work in this domain by releasing the proposed motion dataset both to the public and the market. We conduct a comprehensive study on motion style transfer in the experiment using the state-of-the-art method, and the results show the proposed dataset's validity for the motion style transfer task

    Application of 3D human pose estimation for motion capture and character animation

    Get PDF
    Abstract. Interest in motion capture (mocap) technology is growing every day, and the number of possible applications is multiplying. But such systems are very expensive and are not affordable for personal use. Based on that, this thesis presents the framework that can produce mocap data from regular RGB video and then use it to animate a 3D character according to the movement of the person in the original video. To extract the mocap data from the input video, one of the three 3D pose estimation (PE) methods that are available within the scope of the project is used to determine where the joints of the person in each video frame are located in the 3D space. The 3D positions of the joints are used as mocap data and are imported to Blender which contains a simple 3D character. The data is assigned to the corresponding joints of the character to animate it. To test how the created animation will be working in a different environment, it was imported to the Unity game engine and applied to the native 3D character. The evaluation of the produced animations from Blender and Unity showed that even though the quality of the animation might be not perfect, the test subjects found this approach to animation promising. In addition, during the evaluation, a few issues were discovered and considered for future framework development

    Human motion convolutional autoencoders using different rotation representations

    Get PDF
    This research proposes the application of four different techniques of animation storage (Axis Angle, Quaternions, Rotation Matrices and Euler Angles), in order to determine the advantages and disadvantages of each method through the training and evaluation of autoencoders for reconstructing and denoising parsed data, when passing through a convolutional neural network. The designed autoencoders provide a novel insight into the comparative performance of these animation representation methods in an analog architecture, making them measurable in the same conditions, and thus possible to evaluate with quantitative metrics such as Minimum Square Error (MSE), and Root Mean Square Error (RMSE), as well as qualitatively through close observation of the naturality, its real-time performance after being decoded in full output sequences. My results show that the most accurate method for this purpose qualitatively is Quaternions, followed by Rotation Matrices, Euler Angles and finally with the least accurate results:e Axis Angles. These results persist in decoding and in simple encoding-decoding. Consistent denoising results were achieved in the representations, up until sequences with 25% of added gaussian noise

    Media Data Processing

    Get PDF
    Tato práce popisuje proces tvorby multi-platformního interaktivního systému a demonstruje jeho použitelnost na prototypu aplikace simulující rehabilitaci. Je zde zahrnut SW návrh, 3D programování a produkce animací za použití hloubkového senzoru Kinect. Jádro aplikace bylo vytvořeno v herním enginu Unity 3D a bylo otestováno na Linux Ubuntu, Windows 7, mobilním zařízení Android a Unity Web Playeru. První polovina tohoto dokumentu obsahuje můj průzkum nejmodernějších nástrojů a zdrojů vhodných pro realizaci interaktivní multi-platformní aplikace, pracující s 3D obsahem. Nejprve jsou popsány tři herní enginy (Unity 3D, UDK, Unreal Engine 4), jakožto hlavní vývojová prostředí vhodná k realizaci aplikace. Poté jsou popsány možnosti ukládání dat obecné pro všechny zmíněné enginy - integrovaná úložiště dat, databázové systémy a serializace. Nakonec jsou v práci popsány výhody a nevýhody různých přístupů, jak vytvořit 3D animace. Je zde naznačeno použití 3D softwaru určeného k manuální produkci, ale také různé možnosti snímání pohybu Motion Capture. Druhá část dokumentu popisuje mé zhodnocení informací popsaných v předchozí části a rozhodnutí, které prostředky jsem použil k vytvoření prototypu. Také je zde zkonkretizováno zadání a formulovány základní parametry aplikace. Dále je popsán SW návrh, produkce a zpracování 3D animací a samotná implementace. Nakonec je výsledný prototyp zhodnocen a naznačena rozšíření, které je možné aplikovat v budoucnosti. Co se týče využití samotné aplikace, tato práce předvádí mé nápady, jak mohou být informační technologie využity v oboru fyzioterapie a zdravotnictví obecně. Výsledkem nemělo být nahrazení fyzioterapeutů, nýbrž poskytnutí pomoci jim a jejich pacientům. Tento prototyp je příkladem, který nabízí 3D vizualizaci cvičení a lidského pohybového systému. 3D animace, ukazující, jak by měl být každý cvik správně proveden, mimo jiné, mohou učinit rehabilitace příjemnější a zároveň účinnější.This work describes a process of creating a cross-platform interactive system and it demonstrates usability on prototype application that simulates a rehabilitation session. The application includes 3D visualization possibilities - interactive human muscular system and exercises with 3D animations. The work included SW design, 3D programming and production of animations using Kinect sensor. The core of this prototype was created in Unity 3D game engine. It was deployed and tested on Linux Ubuntu, Windows 7, Android mobile device and Unity Web Player. This work as well manifests my ideas how Information Technology can be used in field of physical therapy.

    Real-time human performance capture and synthesis

    Get PDF
    Most of the images one finds in the media, such as on the Internet or in textbooks and magazines, contain humans as the main point of attention. Thus, there is an inherent necessity for industry, society, and private persons to be able to thoroughly analyze and synthesize the human-related content in these images. One aspect of this analysis and subject of this thesis is to infer the 3D pose and surface deformation, using only visual information, which is also known as human performance capture. Human performance capture enables the tracking of virtual characters from real-world observations, and this is key for visual effects, games, VR, and AR, to name just a few application areas. However, traditional capture methods usually rely on expensive multi-view (marker-based) systems that are prohibitively expensive for the vast majority of people, or they use depth sensors, which are still not as common as single color cameras. Recently, some approaches have attempted to solve the task by assuming only a single RGB image is given. Nonetheless, they can either not track the dense deforming geometry of the human, such as the clothing layers, or they are far from real time, which is indispensable for many applications. To overcome these shortcomings, this thesis proposes two monocular human performance capture methods, which for the first time allow the real-time capture of the dense deforming geometry as well as an unseen 3D accuracy for pose and surface deformations. At the technical core, this work introduces novel GPU-based and data-parallel optimization strategies in conjunction with other algorithmic design choices that are all geared towards real-time performance at high accuracy. Moreover, this thesis presents a new weakly supervised multiview training strategy combined with a fully differentiable character representation that shows superior 3D accuracy. However, there is more to human-related Computer Vision than only the analysis of people in images. It is equally important to synthesize new images of humans in unseen poses and also from camera viewpoints that have not been observed in the real world. Such tools are essential for the movie industry because they, for example, allow the synthesis of photo-realistic virtual worlds with real-looking humans or of contents that are too dangerous for actors to perform on set. But also video conferencing and telepresence applications can benefit from photo-real 3D characters, as they can enhance the immersive experience of these applications. Here, the traditional Computer Graphics pipeline for rendering photo-realistic images involves many tedious and time-consuming steps that require expert knowledge and are far from real time. Traditional rendering involves character rigging and skinning, the modeling of the surface appearance properties, and physically based ray tracing. Recent learning-based methods attempt to simplify the traditional rendering pipeline and instead learn the rendering function from data resulting in methods that are easier accessible to non-experts. However, most of them model the synthesis task entirely in image space such that 3D consistency cannot be achieved, and/or they fail to model motion- and view-dependent appearance effects. To this end, this thesis presents a method and ongoing work on character synthesis, which allow the synthesis of controllable photoreal characters that achieve motion- and view-dependent appearance effects as well as 3D consistency and which run in real time. This is technically achieved by a novel coarse-to-fine geometric character representation for efficient synthesis, which can be solely supervised on multi-view imagery. Furthermore, this work shows how such a geometric representation can be combined with an implicit surface representation to boost synthesis and geometric quality.In den meisten Bildern in den heutigen Medien, wie dem Internet, Büchern und Magazinen, ist der Mensch das zentrale Objekt der Bildkomposition. Daher besteht eine inhärente Notwendigkeit für die Industrie, die Gesellschaft und auch für Privatpersonen, die auf den Mensch fokussierten Eigenschaften in den Bildern detailliert analysieren und auch synthetisieren zu können. Ein Teilaspekt der Anaylse von menschlichen Bilddaten und damit Bestandteil der Thesis ist das Rekonstruieren der 3D-Skelett-Pose und der Oberflächendeformation des Menschen anhand von visuellen Informationen, was fachsprachlich auch als Human Performance Capture bezeichnet wird. Solche Rekonstruktionsverfahren ermöglichen das Tracking von virtuellen Charakteren anhand von Beobachtungen in der echten Welt, was unabdingbar ist für Applikationen im Bereich der visuellen Effekte, Virtual und Augmented Reality, um nur einige Applikationsfelder zu nennen. Nichtsdestotrotz basieren traditionelle Tracking-Methoden auf teuren (markerbasierten) Multi-Kamera Systemen, welche für die Mehrheit der Bevölkerung nicht erschwinglich sind oder auf Tiefenkameras, die noch immer nicht so gebräuchlich sind wie herkömmliche Farbkameras. In den letzten Jahren gab es daher erste Methoden, die versuchen, das Tracking-Problem nur mit Hilfe einer Farbkamera zu lösen. Allerdings können diese entweder die Kleidung der Person im Bild nicht tracken oder die Methoden benötigen zu viel Rechenzeit, als dass sie in realen Applikationen genutzt werden könnten. Um diese Probleme zu lösen, stellt die Thesis zwei monokulare Human Performance Capture Methoden vor, die zum ersten Mal eine Echtzeit-Rechenleistung erreichen sowie im Vergleich zu vorherigen Arbeiten die Genauigkeit von Pose und Oberfläche in 3D weiter verbessern. Der Kern der Methoden beinhaltet eine neuartige GPU-basierte und datenparallelisierte Optimierungsstrategie, die im Zusammenspiel mit anderen algorithmischen Designentscheidungen akkurate Ergebnisse erzeugt und dabei eine Echtzeit-Laufzeit ermöglicht. Daneben wird eine neue, differenzierbare und schwach beaufsichtigte, Multi-Kamera basierte Trainingsstrategie in Kombination mit einem komplett differenzierbaren Charaktermodell vorgestellt, welches ungesehene 3D Präzision erreicht. Allerdings spielt nicht nur die Analyse von Menschen in Bildern in Computer Vision eine wichtige Rolle, sondern auch die Möglichkeit, neue Bilder von Personen in unterschiedlichen Posen und Kamera- Blickwinkeln synthetisch zu rendern, ohne dass solche Daten zuvor in der Realität aufgenommen wurden. Diese Methoden sind unabdingbar für die Filmindustrie, da sie es zum Beispiel ermöglichen, fotorealistische virtuelle Welten mit real aussehenden Menschen zu erzeugen, sowie die Möglichkeit bieten, Szenen, die für den Schauspieler zu gefährlich sind, virtuell zu produzieren, ohne dass eine reale Person diese Aktionen tatsächlich ausführen muss. Aber auch Videokonferenzen und Telepresence-Applikationen können von fotorealistischen 3D-Charakteren profitieren, da diese die immersive Erfahrung von solchen Applikationen verstärken. Traditionelle Verfahren zum Rendern von fotorealistischen Bildern involvieren viele mühsame und zeitintensive Schritte, welche Expertenwissen vorraussetzen und zudem auch Rechenzeiten erreichen, die jenseits von Echtzeit sind. Diese Schritte beinhalten das Rigging und Skinning von virtuellen Charakteren, das Modellieren von Reflektions- und Materialeigenschaften sowie physikalisch basiertes Ray Tracing. Vor Kurzem haben Deep Learning-basierte Methoden versucht, die Rendering-Funktion von Daten zu lernen, was in Verfahren resultierte, die eine Nutzung durch Nicht-Experten ermöglicht. Allerdings basieren die meisten Methoden auf Synthese-Verfahren im 2D-Bildbereich und können daher keine 3D-Konsistenz garantieren. Darüber hinaus gelingt es den meisten Methoden auch nicht, bewegungs- und blickwinkelabhängige Effekte zu erzeugen. Daher präsentiert diese Thesis eine neue Methode und eine laufende Forschungsarbeit zum Thema Charakter-Synthese, die es erlauben, fotorealistische und kontrollierbare 3D-Charakteren synthetisch zu rendern, die nicht nur 3D-konsistent sind, sondern auch bewegungs- und blickwinkelabhängige Effekte modellieren und Echtzeit-Rechenzeiten ermöglichen. Dazu wird eine neuartige Grobzu- Fein-Charakterrepräsentation für effiziente Bild-Synthese von Menschen vorgestellt, welche nur anhand von Multi-Kamera-Daten trainiert werden kann. Daneben wird gezeigt, wie diese explizite Geometrie- Repräsentation mit einer impliziten Oberflächendarstellung kombiniert werden kann, was eine bessere Synthese von geomtrischen Deformationen sowie Bildern ermöglicht.ERC Consolidator Grant 4DRepL

    Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space

    Get PDF
    This major component of the research described in this thesis is 3D computer graphics, specifically the realistic physics-based softbody simulation and haptic responsive environments. Minor components include advanced human-computer interaction environments, non-linear documentary storytelling, and theatre performance. The journey of this research has been unusual because it requires a researcher with solid knowledge and background in multiple disciplines; who also has to be creative and sensitive in order to combine the possible areas into a new research direction. [...] It focuses on the advanced computer graphics and emerges from experimental cinematic works and theatrical artistic practices. Some development content and installations are completed to prove and evaluate the described concepts and to be convincing. [...] To summarize, the resulting work involves not only artistic creativity, but solving or combining technological hurdles in motion tracking, pattern recognition, force feedback control, etc., with the available documentary footage on film, video, or images, and text via a variety of devices [....] and programming, and installing all the needed interfaces such that it all works in real-time. Thus, the contribution to the knowledge advancement is in solving these interfacing problems and the real-time aspects of the interaction that have uses in film industry, fashion industry, new age interactive theatre, computer games, and web-based technologies and services for entertainment and education. It also includes building up on this experience to integrate Kinect- and haptic-based interaction, artistic scenery rendering, and other forms of control. This research work connects all the research disciplines, seemingly disjoint fields of research, such as computer graphics, documentary film, interactive media, and theatre performance together.Comment: PhD thesis copy; 272 pages, 83 figures, 6 algorithm
    corecore