RotationNet: Joint Object Categorization and Pose Estimation Using Multiviews from Unsupervised Viewpoints

We propose a Convolutional Neural Network (CNN)-based model "RotationNet," which takes multi-view images of an object as input and jointly estimates its pose and object category. Unlike previous approaches that use known viewpoint labels for training, our method treats the viewpoint labels as latent variables, which are learned in an unsupervised manner during the training using an unaligned object dataset. RotationNet is designed to use only a partial set of multi-view images for inference, and this property makes it useful in practical scenarios where only partial views are available. Moreover, our pose alignment strategy enables one to obtain view-specific feature representations shared across classes, which is important to maintain high accuracy in both object categorization and pose estimation. Effectiveness of RotationNet is demonstrated by its superior performance to the state-of-the-art methods of 3D object classification on 10- and 40-class ModelNet datasets. We also show that RotationNet, even trained without known poses, achieves the state-of-the-art performance on an object pose estimation dataset. The code is available on https://github.com/kanezaki/rotationnetComment: 24 pages, 23 figures. Accepted to CVPR 201

ISB clinical biomechanics award winner 2021: Tibio-femoral kinematics of natural versus replaced knees - A comparison using dynamic videofluoroscopy

BACKGROUND A comparison of natural versus replaced tibio-femoral kinematics in vivo during challenging activities of daily living can help provide a detailed understanding of the mechanisms leading to unsatisfactory results and lay the foundations for personalised implant selection and surgical implantation, but also enhance further development of implant designs towards restoring physiological knee function. The aim of this study was to directly compare in vivo tibio-femoral kinematics in natural versus replaced knees throughout complete cycles of different gait activities using dynamic videofluoroscopy. METHODS Twenty-seven healthy and 30 total knee replacement subjects (GMK Sphere, GMK PS, GMK UC) were assessed during multiple complete gait cycles of level walking, downhill walking, and stair descent using dynamic videofluoroscopy. Following 2D/3D registration, tibio-femoral rotations, condylar antero-posterior translations, and the location of the centre of rotation were compared. FINDINGS The total knee replacement groups predominantly experienced reduced tibial internal/external rotation and altered medial and lateral condylar antero-posterior translations compared to natural knees. An average medial centre of rotation was found for the natural and GMK sphere groups in all three activities, whereas the GMK PS and UC groups experienced a more central to lateral centre of rotation. INTERPRETATION Each total knee replacement design exhibited characteristic motion patterns, with the GMK Sphere most closely replicating the medial centre of rotation found for natural knees. Despite substantial similarities between the subject groups, none of the implant geometries was able to replicate all aspects of natural tibio-femoral kinematics, indicating that different implant geometries might best address individual functional needs

Capturing natural-colour 3D models of insects for species discovery

Collections of biological specimens are fundamental to scientific understanding and characterization of natural diversity. This paper presents a system for liberating useful information from physical collections by bringing specimens into the digital domain so they can be more readily shared, analyzed, annotated and compared. It focuses on insects and is strongly motivated by the desire to accelerate and augment current practices in insect taxonomy which predominantly use text, 2D diagrams and images to describe and characterize species. While these traditional kinds of descriptions are informative and useful, they cannot cover insect specimens "from all angles" and precious specimens are still exchanged between researchers and collections for this reason. Furthermore, insects can be complex in structure and pose many challenges to computer vision systems. We present a new prototype for a practical, cost-effective system of off-the-shelf components to acquire natural-colour 3D models of insects from around 3mm to 30mm in length. Colour images are captured from different angles and focal depths using a digital single lens reflex (DSLR) camera rig and two-axis turntable. These 2D images are processed into 3D reconstructions using software based on a visual hull algorithm. The resulting models are compact (around 10 megabytes), afford excellent optical resolution, and can be readily embedded into documents and web pages, as well as viewed on mobile devices. The system is portable, safe, relatively affordable, and complements the sort of volumetric data that can be acquired by computed tomography. This system provides a new way to augment the description and documentation of insect species holotypes, reducing the need to handle or ship specimens. It opens up new opportunities to collect data for research, education, art, entertainment, biodiversity assessment and biosecurity control.Comment: 24 pages, 17 figures, PLOS ONE journa