A Preliminary Investigation into a Deep Learning Implementation for Hand Tracking on Mobile Devices

Hand tracking is an essential component of computer graphics and human-computer interaction applications. The use of RGB camera without specific hardware and sensors (e.g., depth cameras) allows developing solutions for a plethora of devices and platforms. Although various methods were proposed, hand tracking from a single RGB camera is still a challenging research area due to occlusions, complex backgrounds, and various hand poses and gestures. We present a mobile application for 2D hand tracking from RGB images captured by the smartphone camera. The images are processed by a deep neural network, modified specifically to tackle this task and run on mobile devices, looking for a compromise between performance and computational time. Network output is used to show a 2D skeleton on the user's hand. We tested our system on several scenarios, showing an interactive hand tracking level and achieving promising results in the case of variable brightness and backgrounds and small occlusions

Overview of the treatment of historical industrial heritage in engineering graphics

This article presents a comprehensive review of the stages of research when addressing the issue of historical industrial heritage from the point view of engineering graphics, looking at its influence on all stages of the value chain. It provides a contextual introduction, explaining the research, the aim pursued and the priority objectives of this type of work. It also presents the methodology and the materials used, specifying important details needed to obtain a realistic result, such as fieldwork, 3D virtual modeling, virtual reconstruction, and more usual techniques used such as augmented reality and virtual reality. Finally, the dissemination of the results and the advantages and disadvantages of the virtual reconstructions are presented

Game engines selection framework for high-fidelity serious applications

Serious games represent the state-of-the-art in the convergence of electronic gaming technologies with instructional design principles and pedagogies. Despite the value of high-fidelity content in engaging learners and providing realistic training environments, building games which deliver high levels of visual and functional realism is a complex, time consuming and expensive process. Therefore, commercial game engines, which provide a development environment and resources to more rapidly create high-fidelity virtual worlds, are increasingly used for serious as well as for entertainment applications. Towards this intention, the authors propose a new framework for the selection of game engines for serious applications and sets out five elements for analysis of engines in order to create a benchmarking approach to the validation of game engine selection. Selection criteria for game engines and the choice of platform for Serious Games are substantially different from entertainment games, as Serious Games have very different objectives, emphases and technical requirements. In particular, the convergence of training simulators with serious games, made possible by increasing hardware rendering capacity is enabling the creation of high-fidelity serious games, which challenge existing instructional approaches. This paper overviews several game engines that are suitable for high-fidelity serious games, using the proposed framework

Analysis of the hands in egocentric vision: A survey

Egocentric vision (a.k.a. first-person vision - FPV) applications have thrived over the past few years, thanks to the availability of affordable wearable cameras and large annotated datasets. The position of the wearable camera (usually mounted on the head) allows recording exactly what the camera wearers have in front of them, in particular hands and manipulated objects. This intrinsic advantage enables the study of the hands from multiple perspectives: localizing hands and their parts within the images; understanding what actions and activities the hands are involved in; and developing human-computer interfaces that rely on hand gestures. In this survey, we review the literature that focuses on the hands using egocentric vision, categorizing the existing approaches into: localization (where are the hands or parts of them?); interpretation (what are the hands doing?); and application (e.g., systems that used egocentric hand cues for solving a specific problem). Moreover, a list of the most prominent datasets with hand-based annotations is provided

How to Obtain an Orthodontic Virtual Patient through Superimposition of Three-Dimensional Data: A Systematic Review

Background: This systematic review summarizes the current knowledge on the superimposition of three-dimensional (3D) diagnostic records to realize an orthodontic virtual patient. The aim of this study is to analyze the accuracy of the state-of-the-art digital workflow. Methods: The research was carried out by an electronic and manual query eectuated from ISS (Istituto Superiore di Sanit\ue0 in Rome) on three dierent databases (MEDLINE, Cochrane Library and ISI WEB OF SCIENCE) up to 31st January 2020. The search focused on studies that superimposed at least two dierent 3D records to build up a 3D virtual patient\u2014information about the devices used to acquire 3D data, the software used to match data and the superimposition method applied have been summarized. Results: 1374 titles were retrieved from the electronic search. After title-abstract screening, 65 studies were selected. After full-text analysis, 21 studies were included in the review. Dierent 3D datasets were used: facial skeleton (FS), extraoral soft tissues (ST) and dentition (DENT). The information provided by the 3D data was superimposed in four dierent combinations: FS + DENT (13 papers), FS + ST (5 papers), ST + DENT (2 papers) and all the types (FS + ST + DENT) (1 paper). Conclusions: The surface-based method was most frequently used for 3D objects superimposition (11 papers), followed by the point-based method (6 papers), with or without fiducial markers, and the voxel-based method (1 paper). Most of the papers analyzed the accuracy of the superimposition procedure (15 papers), while the remaining were proof-of-principles (10 papers) or compared dierent methods (3 papers). Further studies should focus on the definition of a gold standard. The patient is going to have a huge advantage from complete digital planning when more information about the spatial relationship of anatomical structures are needed: ectopic, impacted and supernumerary teeth, root resorption and angulations, cleft lip and palate (CL/P), alveolar boundary conditions, periodontally compromised patients, temporary anchorage devices (TADs), maxillary transverse deficiency, airway analyses, obstructive sleep apnea (OSAS), TMJ disorders and orthognathic and cranio-facial surgery

Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space

This major component of the research described in this thesis is 3D computer graphics, specifically the realistic physics-based softbody simulation and haptic responsive environments. Minor components include advanced human-computer interaction environments, non-linear documentary storytelling, and theatre performance. The journey of this research has been unusual because it requires a researcher with solid knowledge and background in multiple disciplines; who also has to be creative and sensitive in order to combine the possible areas into a new research direction. [...] It focuses on the advanced computer graphics and emerges from experimental cinematic works and theatrical artistic practices. Some development content and installations are completed to prove and evaluate the described concepts and to be convincing. [...] To summarize, the resulting work involves not only artistic creativity, but solving or combining technological hurdles in motion tracking, pattern recognition, force feedback control, etc., with the available documentary footage on film, video, or images, and text via a variety of devices [....] and programming, and installing all the needed interfaces such that it all works in real-time. Thus, the contribution to the knowledge advancement is in solving these interfacing problems and the real-time aspects of the interaction that have uses in film industry, fashion industry, new age interactive theatre, computer games, and web-based technologies and services for entertainment and education. It also includes building up on this experience to integrate Kinect- and haptic-based interaction, artistic scenery rendering, and other forms of control. This research work connects all the research disciplines, seemingly disjoint fields of research, such as computer graphics, documentary film, interactive media, and theatre performance together.Comment: PhD thesis copy; 272 pages, 83 figures, 6 algorithm

Linking Physical Objects to Their Digital Twins via Fiducial Markers Designed for Invisibility to Humans

The ability to label and track physical objects that are assets in digital representations of the world is foundational to many complex systems. Simple, yet powerful methods such as bar- and QR-codes have been highly successful, e.g. in the retail space, but the lack of security, limited information content and impossibility of seamless integration with the environment have prevented a large-scale linking of physical objects to their digital twins. This paper proposes to link digital assets created through building information modeling (BIM) with their physical counterparts using fiducial markers with patterns defined by cholesteric spherical reflectors (CSRs), selective retroreflectors produced using liquid crystal self-assembly. The markers leverage the ability of CSRs to encode information that is easily detected and read with computer vision while remaining practically invisible to the human eye. We analyze the potential of a CSR-based infrastructure from the perspective of BIM, critically reviewing the outstanding challenges in applying this new class of functional materials, and we discuss extended opportunities arising in assisting autonomous mobile robots to reliably navigate human-populated environments, as well as in augmented reality

Image processing techniques for mixed reality and biometry

2013 - 2014This thesis work is focused on two applicative fields of image processing research, which, for different reasons, have become particularly active in the last decade: Mixed Reality and Biometry. Though the image processing techniques involved in these two research areas are often different, they share the key objective of recognizing salient features typically captured through imaging devices. Enabling technologies for augmented/mixed reality have been improved and refined throughout the last years and more recently they seems to have finally passed the demo stage to becoming ready for practical industrial and commercial applications. To this regard, a crucial role will likely be played by the new generation of smartphones and tablets, equipped with an arsenal of sensors connections and enough processing power for becoming the most portable and affordable AR platform ever. Within this context, techniques like gesture recognition by means of simple, light and robust capturing hardware and advanced computer vision techniques may play an important role in providing a natural and robust way to control software applications and to enhance onthe- field operational capabilities. The research described in this thesis is targeted toward advanced visualization and interaction strategies aimed to improve the operative range and robustness of mixed reality applications, particularly for demanding industrial environments... [edited by Author]XIII n.s