65 research outputs found
Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space
This major component of the research described in this thesis is 3D computer
graphics, specifically the realistic physics-based softbody simulation and
haptic responsive environments. Minor components include advanced
human-computer interaction environments, non-linear documentary storytelling,
and theatre performance. The journey of this research has been unusual because
it requires a researcher with solid knowledge and background in multiple
disciplines; who also has to be creative and sensitive in order to combine the
possible areas into a new research direction. [...] It focuses on the advanced
computer graphics and emerges from experimental cinematic works and theatrical
artistic practices. Some development content and installations are completed to
prove and evaluate the described concepts and to be convincing. [...] To
summarize, the resulting work involves not only artistic creativity, but
solving or combining technological hurdles in motion tracking, pattern
recognition, force feedback control, etc., with the available documentary
footage on film, video, or images, and text via a variety of devices [....] and
programming, and installing all the needed interfaces such that it all works in
real-time. Thus, the contribution to the knowledge advancement is in solving
these interfacing problems and the real-time aspects of the interaction that
have uses in film industry, fashion industry, new age interactive theatre,
computer games, and web-based technologies and services for entertainment and
education. It also includes building up on this experience to integrate Kinect-
and haptic-based interaction, artistic scenery rendering, and other forms of
control. This research work connects all the research disciplines, seemingly
disjoint fields of research, such as computer graphics, documentary film,
interactive media, and theatre performance together.Comment: PhD thesis copy; 272 pages, 83 figures, 6 algorithm
SimSwap: An Efficient Framework For High Fidelity Face Swapping
We propose an efficient framework, called Simple Swap (SimSwap), aiming for
generalized and high fidelity face swapping. In contrast to previous approaches
that either lack the ability to generalize to arbitrary identity or fail to
preserve attributes like facial expression and gaze direction, our framework is
capable of transferring the identity of an arbitrary source face into an
arbitrary target face while preserving the attributes of the target face. We
overcome the above defects in the following two ways. First, we present the ID
Injection Module (IIM) which transfers the identity information of the source
face into the target face at feature level. By using this module, we extend the
architecture of an identity-specific face swapping algorithm to a framework for
arbitrary face swapping. Second, we propose the Weak Feature Matching Loss
which efficiently helps our framework to preserve the facial attributes in an
implicit way. Extensive experiments on wild faces demonstrate that our SimSwap
is able to achieve competitive identity performance while preserving attributes
better than previous state-of-the-art methods. The code is already available on
github: https://github.com/neuralchen/SimSwap.Comment: Accepted by ACMMM 202
Visual-auditory visualisation of dynamic multi-scale heterogeneous objects.
The multi-scale phenomena analysis is an area of active research that is connecting simulations with experiments to get a correct insight into the compound dynamic structure. Visualisation is a challenging task due to a large amount of data and a wide range of complex data representations. The analysis of dynamic multi-scale phenomena requires a combination of geometric modelling and rendering techniques for the analysis of the changes in the internal structure in the case of data coming from different sources of various nature. Moreover, the area often addresses the limitations of solely visual data representation and considers the introduction of other sensory stimuli as a well-known tool to enhance visual analysis. However, there is a lack of software tools allowing perform an advanced real-time analysis of heterogeneous phenomena properties. The hardware-accelerated volume rendering allows getting insight into the internal structure of complex multi-scale phenomena. The technique is convenient for detailed visual analysis and highlights the features of interest in complex structures and is an area of active research. However, the conventional volume visualisation is limited to the use of transfer functions that operate on homogeneous material and, as a result, does not provide flexibility in geometry and material distribution modelling that is crucial for the analysis of heterogeneous objects. Moreover, the extension to visual-auditory analysis emphasises the necessity to review the entire conventional volume visualisation pipeline. The multi-sensory feedback highly depends on the use of modern hardware and software advances for real-time modelling and evaluation. In this work, we explore the aspects of the design of visual-auditory pipelines for the analysis of dynamic multi-scale properties of heterogeneous objects that can allow overcoming well-known problems of complex representations solely visual analysis. We consider the similarities between light and sound propagation as a solution to the problem. The approach benefits from a combination of GPU accelerated ray-casting, geometry, optical and auditory properties modelling. We discuss how the modern GPU techniques application in those areas allows introducing a unified approach to the visual-auditory analysis of dynamic multi-scale heterogeneous objects. Similarly to the conventional volume rendering technique based on light propagation, we model auditory feedback as a result of initial impulse propagation through 3D space and its digital representation as a sampled sound wave obtained with the ray-casting procedure. The auditory stimuli can complement visual ones in the analysis of the dynamic multi-scale heterogeneous object. We propose a framework that facilitates the design of dynamic multi-scale heterogeneous objects visual-auditory pipeline and discuss the framework application for two case studies. The first is a molecular phenomena study that is a result of molecular dynamics simulation and quantum simulation. The second explores microstructures in digital fabrication with an arbitrary irregular lattice structure. For considered case studies, the visual-auditory techniques facilitate the interactive analysis of both spatial structure and internal multi-scale properties of volume nature in complex heterogeneous objects. A GPU-accelerated framework for visual-auditory analysis of heterogeneous objects can be applied and extend beyond this research. Thus, to specify the main direction of such extension from the point of view of the potential users, strengthen the value of this research as well as to evaluate the vision of the application of the techniques described above, we carry out a preliminary evaluation. The user study aims to compare our expectations on the visual-auditory approach with the views of the potential users of this system if it is implemented as a software product. A preliminary evaluation study was carried out with limitations imposed by 2020/2021 restrictions. However, it confirms that the main direction for the visual-auditory analysis of heterogeneous objects has been identified correctly and visual and auditory stimuli can complement each other in the analysis of both volume and spatial distribution properties of heterogeneous phenomena. The user reviews also highlight the necessary enhancements that should be introduced to the approach in terms of the design of more complex user interfaces and consideration of additional application cases. To provide a more detailed picture on evaluation results and recommendations introduced, we also identify the key factors that define the user vision of the approach further enhancement and its possible application areas, such as users experience in the area of complex physical phenomena analysis or multi-sensory area. The discussed in this work aspects of heterogeneous objects analysis task, theoretical and practical solutions allow considering the application, further development and enhancement of the results in multidisciplinary areas of GPU accelerated High-performance visualisation pipelines design and multi-sensory analysis
Languages of games and play: A systematic mapping study
Digital games are a powerful means for creating enticing, beautiful, educational, and often highly addictive interactive experiences that impact the lives of billions of players worldwide. We explore what informs the design and construction of good games to learn how to speed-up game development. In particular, we study to what extent languages, notations, patterns, and tools, can offer experts theoretical foundations, systematic techniques, and practical solutions they need to raise their productivity and improve the quality of games and play. Despite the growing number of publications on this topic there is currently no overview describing the state-of-the-art that relates research areas, goals, and applications. As a result, efforts and successes are often one-off, lessons learned go overlooked, language reuse remains minimal, and opportunities for collaboration and synergy are lost. We present a systematic map that identifies relevant publications and gives an overview of research areas and publication venues. In addition, we categorize research perspectives along common objectives, techniques, and approaches, illustrated by summaries of selected languages. Finally, we distill challenges and opportunities for future research and development
Does modality make a difference? A comparative study of mobile augmented reality for education and training
Includes bibliographical references.2022 Fall.As augmented reality (AR) technologies progress they have begun to impact the field of education and training. Many prior studies have explored the potential benefits and challenges to integrating emerging technologies into educational practices. Both internal and external factors may impact the overall adoption of the technology, however there are key benefits identified for the schema building process, which is important for knowledge acquisition. This study aims to elaborate and expand upon prior studies to explore the question does mobile augmented reality provide for stronger knowledge retention compared to other training and education modalities? To address this question this study takes a comparative experimental approach by exposing participants to one of three training modalities (AR, paper manual, or online video) and evaluating their knowledge retention and other educational outcomes
A Cloud-Based Extensible Avatar For Human Robot Interaction
Adding an interactive avatar to a human-robot interface requires the development of tools that animate the avatar so as to simulate an intelligent conversation partner. Here we describe a toolkit that supports interactive avatar modeling for human-computer interaction. The toolkit utilizes cloud-based speech-to-text software that provides active listening, a cloud-based AI to generate appropriate textual responses to user queries, and a cloud-based text-to-speech generation engine to generate utterances for this text. This output is combined with a cloud-based 3D avatar animation synchronized to the spoken response. Generated text responses are embedded within an XML structure that allows for tuning the nature of the avatar animation to simulate different emotional states. An expression package controls the avatar's facial expressions. The introduced rendering latency is obscured through parallel processing and an idle loop process that animates the avatar between utterances. The efficiency of the approach is validated through a formal user study
Additive manufacturing in bespoke interactive devices-a thematic analysis
Additive Manufacturing (AM) facilitates product development due to the various native advantages of AM when compared to traditional manufacturing processes. Efficiency, customisation, innovation, and ease of product modifications are a few advantages of AM. This manufacturing process can therefore be applied to fabricate customisable devices, such as bespoke interactive devices for rehabilitation purposes. In this context, a two-day workshop titled Design for Additive Manufacturing: Future Interactive Devices (DEFINED) was held to discuss the design for AM issues encountered in the development of an innovative bespoke controller and supporting platform, in a Virtual Reality (VR)-based environment, intended for people with limited dexterity in their hands. The workshop sessions were transcribed, and a thematic analysis was carried out to identify the main topics discussed. The themes were Additive Manufacturing, Generative Design Algorithms, User-Centred Design, Measurement Devices for Data Acquisition, Virtual Reality, Augmented Reality, and Haptics. These themes were then discussed in relation to the available literature. The main conclusion of this workshop was that a coherent design for AM tools is needed by designers to take AM considerations throughout the design process, since they lack the AM knowledge required to develop bespoke interactive devices
Game-Based Learning, Gamification in Education and Serious Games
The aim of this book is to present and discuss new advances in serious games to show how they could enhance the effectiveness and outreach of education, advertising, social awareness, health, policies, etc. We present their use in structured learning activities, not only with a focus on game-based learning, but also on the use of game elements and game design techniques to gamify the learning process. The published contributions really demonstrate the wide scope of application of game-based approaches in terms of purpose, target groups, technologies and domains and one aspect they have in common is that they provide evidence of how effective serious games, game-based learning and gamification can be
- …