In daily life, humans utilize hands to manipulate objects. Modeling the shape
of objects that are manipulated by the hand is essential for AI to comprehend
daily tasks and to learn manipulation skills. However, previous approaches have
encountered difficulties in reconstructing the precise shapes of hand-held
objects, primarily owing to a deficiency in prior shape knowledge and
inadequate data for training. As illustrated, given a particular type of tool,
such as a mug, despite its infinite variations in shape and appearance, humans
have a limited number of 'effective' modes and poses for its manipulation. This
can be attributed to the fact that humans have mastered the shape prior of the
'mug' category, and can quickly establish the corresponding relations between
different mug instances and the prior, such as where the rim and handle are
located. In light of this, we propose a new method, CHORD, for Category-level
Hand-held Object Reconstruction via shape Deformation. CHORD deforms a
categorical shape prior for reconstructing the intra-class objects. To ensure
accurate reconstruction, we empower CHORD with three types of awareness:
appearance, shape, and interacting pose. In addition, we have constructed a new
dataset, COMIC, of category-level hand-object interaction. COMIC contains a
rich array of object instances, materials, hand interactions, and viewing
directions. Extensive evaluation shows that CHORD outperforms state-of-the-art
approaches in both quantitative and qualitative measures. Code, model, and
datasets are available at https://kailinli.github.io/CHORD.Comment: To be presented at ICCV 2023, Pari