Novel-view Synthesis and Pose Estimation for Hand-Object Interaction from Sparse Views

Hand-object interaction understanding and the barely addressed novel view synthesis are highly desired in the immersive communication, whereas it is challenging due to the high deformation of hand and heavy occlusions between hand and object. In this paper, we propose a neural rendering and pose estimation system for hand-object interaction from sparse views, which can also enable 3D hand-object interaction editing. We share the inspiration from recent scene understanding work that shows a scene specific model built beforehand can significantly improve and unblock vision tasks especially when inputs are sparse, and extend it to the dynamic hand-object interaction scenario and propose to solve the problem in two stages. We first learn the shape and appearance prior knowledge of hands and objects separately with the neural representation at the offline stage. During the online stage, we design a rendering-based joint model fitting framework to understand the dynamic hand-object interaction with the pre-built hand and object models as well as interaction priors, which thereby overcomes penetration and separation issues between hand and object and also enables novel view synthesis. In order to get stable contact during the hand-object interaction process in a sequence, we propose a stable contact loss to make the contact region to be consistent. Experiments demonstrate that our method outperforms the state-of-the-art methods. Code and dataset are available in project webpage https://iscas3dv.github.io/HO-NeRF

An Integrating Framework for Mixed Systems

International audienceTechnological advances in hardware manufacturing led to an extended range of possibilities for designing physical-digital objects involved in a mixed system. Mixed systems can take various forms and include augmented reality, augmented virtuality, and tangible systems. In this very dynamic context, it is difficult to compare existing mixed systems and to systematically explore the design space. Addressing this design problem, this chapter presents a unified point of view on mixed systems by focusing on mixed objects involved in interaction, i.e. hybrid physical-digital objects straddling physical and digital worlds. Our integrating framework is made of two complementary facets of a mixed object: we define intrinsic characteristics of an object as well as extrinsic characteristics of an object by considering its role in the interaction. Such characteristics of an object are useful for comparing existing mixed systems at a fine-grain level. The taxonomic power of these characteristics is discussed in the context of existing mixed systems from the literature. Their generative power is illustrated by considering a system, Roam, which we designed and developed

An Integrating Framework for Mixed Systems

International audienceTechnological advances in hardware manufacturing led to an extended range of possibilities for designing physical-digital objects involved in a mixed system. Mixed systems can take various forms and include augmented reality, augmented virtuality, and tangible systems. In this very dynamic context, it is difficult to compare existing mixed systems and to systematically explore the design space. Addressing this design problem, this chapter presents a unified point of view on mixed systems by focusing on mixed objects involved in interaction, i.e. hybrid physical-digital objects straddling physical and digital worlds. Our integrating framework is made of two complementary facets of a mixed object: we define intrinsic characteristics of an object as well as extrinsic characteristics of an object by considering its role in the interaction. Such characteristics of an object are useful for comparing existing mixed systems at a fine-grain level. The taxonomic power of these characteristics is discussed in the context of existing mixed systems from the literature. Their generative power is illustrated by considering a system, Roam, which we designed and developed

A Comparative Analysis of Structured and Object-Oriented Programming Methods

The concepts of structured and object-oriented programming methods are not relatively new but these approaches are still very much useful and relevant in today’s programming paradigm. In this paper, wedistinguish the features of structured programs from that of object oriented programs. Structured programming is a method of organizing and coding programs that can provide easy understanding and modification, whereas objectoriented programming (OOP) consists of a set of objects, which can vary dynamically, and which can execute byacting and reacting to each other, in much the same way that a real-world process proceeds (the interaction of realworldobjects). An object-oriented approach makes programs more intuitive to design, faster to develop, more amenable to modifications, and easier to understand. With the traditional, procedural-oriented/structuredprogramming, a program describes a series of steps to be performed (an algorithm). In the object-oriented view of programming, instead of programs consisting of sets of data loosely coupled to many different procedures, objectoriented programs consist of software modules called objects that encapsulate both data and processing while hidingtheir inner complexities from programmers and hence from other object

A Comparative Analysis of Structured and Object-Oriented Programming Methods

The concepts of structured and object-oriented programming methods are not relatively new but these approaches are still very much useful and relevant in today’s programming paradigm. In this paper, wedistinguish the features of structured programs from that of object oriented programs. Structured programming is a method of organizing and coding programs that can provide easy understanding and modification, whereas objectoriented programming (OOP) consists of a set of objects, which can vary dynamically, and which can execute byacting and reacting to each other, in much the same way that a real-world process proceeds (the interaction of realworldobjects). An object-oriented approach makes programs more intuitive to design, faster to develop, more amenable to modifications, and easier to understand. With the traditional, procedural-oriented/structuredprogramming, a program describes a series of steps to be performed (an algorithm). In the object-oriented view of programming, instead of programs consisting of sets of data loosely coupled to many different procedures, objectoriented programs consist of software modules called objects that encapsulate both data and processing while hidingtheir inner complexities from programmers and hence from other object

The Investigation on Using Unity3D Game Engine in Urban Design Study

Developing a virtual 3D environment by using game engine is a strategy to incorporate various multimedia data into one platform. The characteristic of game engine that is preinstalled with interactive and navigation tools allows users to explore and engage with the game objects. However, most CAD and GIS applications are not equipped with 3D tools and navigation systems intended to the user experience. In particular, 3D game engines provide standard 3D navigation tools as well as any programmable view to create engaging navigation thorough the virtual environment. By using a game engine, it is possible to create other interaction such as object manipulation, non playing character (NPC) interaction with player and/or environment. We conducted analysis on previous game engines and experiment on urban design project with Unity3D game engine for visualization and interactivity. At the end, we present the advantages and limitations using game technology as visual representation tool for architecture and urban design studies

Designer-User Interaction as the Core of the Design & IT Innovation Process: A Socio-Cultural Perspective

In this paper, a model of designer-user interaction as a socio-cultural phenomenon is proposed with the following question: how do the changes in the designer\u27s perspective on the user\u27s physical and social experiences lead to design refinement or design innovation sequences? Adopting Bourdieu’s theory of practice as a macro view, we interpret field as rules of action, habitus as modes of action, and practice as situated actions in the design process. Particularly, this research argues that the changes in the designer’s habitus, as a result of newly acquired knowledge from user research, entail innovation of practice and expansion of field. In addition, the concept of boundary object is considered how the designer’s research activities assist them in acquiring knowledge from various sources, and to translate / transform it across domain boundaries during the process. Four case studies are presented as empirical evidence

Perancangan Sistem Informasi untuk Mendukung Sistem Peringatan Dini suatu Manajemen Persediaan pada Perusahaan Grosir

Abstract In this paper, we concerned about improving inefficiences of the business processes at store room areagrocery products in a grocery company. Due to inefficiencies of the inventory management, it is necessary to improve them related with providing good information and data management. The main problem is miss information related with early warning system of lifetime products. They often have unsold inventory grocery products as impact of expired date. This journal analyzes a system to find out the need for system, then designs the system based on the system requirement obtained from the analysis. The information system of store room area grocery product is designed by using an object-oriented approach and UML modeling language to identify the actor, to make the system use case, to model the business process using the activity chart, to make interaction scheme using sequence diagram and to identify class. In the designing stage, this database and interface design is made. After database and interface design, the next stage is to make the program code by considering the use case and interaction diagrams that are made previously in modeling the object-oriented system. The result of this research consists of input constituting login form, product, suppliers, unit, category, action, location, and personnel data, addition, withdrawal, and action transactions, searching for product data, personnel data, location data and report input. Meanwhile the output constitutes withdrawal list, increment, product early earning reports approaching the expired date, product list, location list, personnel list, stock op name checklist, and the product master label. The outputs of the information system are reports and documents in monitor screen view and in printing. The validation showed that the information system designed can support the early warning system of expired date of grocery products in store room area. Keywords: information system, object oriented, early warning, inventory management, expired dat