Viewpoint Interpolation: Direct and Variational Methods

International audienceWe address the topic of novel view synthesis from a stereoscopic pair of images. The techniques have mainly 3 stages: the reconstruction of correspondences between the views, the estimation of the blending factor of each view for the final view, and the rendering. The state of the art has mainly focused on the correspondence topic, but little work addresses the question of which blending factors are best. The rendering methods can be classified into "direct" methods, defining the final image as a function of the original images, and "variational" methods, where the synthesized image is expressed as the solution minimising an energy. In this paper, we experiment different combinations of the blending factors and the rendering method, in order to demonstrate the effect of these two factors on the final image quality

Interpolation de points de vue : approches directe et variationnelle

National audienceNous abordons la problématique d'interpolation de points de vue à partir d'une paire d'images stéréoscopique. Ces techniques comportent généralement 3 étapes : l'estimation des correspondances entre les vues, le dosage des contributions de chaque image dans la vue finale, et le rendu. D'un côté, tandis que l'état de l'art est très vaste dans l'estimation des correspondances, nous trouvons peu de travaux formels analysant quel est le "bon" dosage des contributions lors du mélange des images. D'un autre côté, concernant le rendu de nouveaux points de vue nous identifions deux groupes de méthodes bien distincts, les méthodes "directes", et les méthodes "variationnelles". Nous conduisons une étude pour analyser la performance des facteurs de dosage ainsi que l'impact de la méthode utilisée sur le résultat final obtenu. Nous évaluons ces méthodes sur des scènes lambertiennes et non-lambertiennes afin de voir, dans chaque cas, quel choix est le plus pertinent

Dynamic Stereoscopic Previz

International audienceThe pre-production stage in a film workflow is important to save time during production. To be useful in stereoscopic 3-D movie-making, storyboards and previz tools need to be adapted in at least two ways. First, it should be possible to specify the desired depth values with suitable and intuitive user interfaces. Second, it should be possible to preview the stereoscopic movie with a suitable screen size. In this paper, we describe a novel technique for simulating a cinema projection room with arbitrary dimensions in a real-time game engine, while controling the camera interaxial and convergence parameters with a gamepad controller. Our technique has been implemented in the Blender Game Engine and tested during the shooting of a short movie. Qualitative experimental results show that our technique overcomes the limitations of previous work in stereoscopic previz and can usefully complement traditional storyboards during pre-production of stereoscopic 3-D movies

Détection de différence de mise au point lors des prises de vues stéréoscopiques

National audienceLive-action stereoscopic content production requires a stereo rig with two cameras precisely matched and aligned. While most deviations from this perfect setup can be corrected either live or in post-production, a difference in the focus distance or focus range between the two cameras will lead to unrecoverable degradations of the stereoscopic footage. In this paper we detect focus mismatch between views of a stereoscopic pair in four steps. First, we compute a dense disparity map. Then, we use a measure to compare focus in both images. After this, we use robust statistics to find which images' zones have a different focus. Finally, to give useful feedback, we show the results on the original images and give hints on how to solve the focus mismatch.La production d'images stéréoscopiques nécessite un rig stéréoscopique avec deux caméras parfaitement synchronisées et alignées. La plupart des imprécisions de ce montage peuvent être corrigées en direct ou en post-production. Par contre, une différence de distance de mise au point ou de profondeur de champ entre les caméras produira des dégradations ir-récupérables dans les images. Dans cet article nous détectons des différences de mise au point entre les deux vues d'une paire stéréoscopique en quatre étapes. D'abord nous calculons une carte de disparité dense. Ensuite nous mesurons la netteté dans chaque image et nous com-parons ces mesures. Puis, avec des méthodes statistiques robustes, nous identifions les zones de l'image qui présentent des différences. Finalement, nous proposons une méthode de visua-lisation sur les images originales pour informer l'opérateur des problèmes, et lui donner des indices pour les résoudre

OSSO: Obtaining Skeletal Shape from Outside

International audienceWe address the problem of inferring the anatomic skeleton of a person, in an arbitrary pose, from the 3D surface of the body; i.e. we predict the inside (bones) from the outside (skin). This has many applications in medicine and biomechanics. Existing state-of-the-art biomechanical skeletons are detailed but do not easily generalize to new subjects. Additionally, computer vision and graphics methods that predict skeletons are typically heuristic, not learned from data, do not leverage the full 3D body surface, and are not validated against ground truth. To our knowledge, our system, called OSSO (Obtaining Skeletal Shape from Outside), is the first to learn the mapping from the 3D body surface to the internal skeleton from real data. We do so using 1000 male and 1000 female dual-energy X-ray absorptiometry (DXA) scans. To these, we fit a parametric 3D body shape model (STAR) to capture the body surface and a novel part-based 3D skeleton model to capture the bones. This provides inside/outside training pairs. We model the statistical variation of full skeletons using PCA in a pose-normalized space. We then train a regressor from body shape parameters to skeleton shape parameters and refine the skeleton to satisfy constraints on physical plausibility. Given an arbitrary 3D body shape and pose, OSSO predicts a realistic skeleton inside. In contrast to previous work, we evaluate the accuracy of the skeleton shape quantitatively on held out DXA scans, outperforming the state-of-the art. We also show 3D skeleton prediction from varied and challenging 3D bodies. The code to infer a skeleton from a body shape is available for research at https://osso.is.tue.mpg.de/, and the dataset of paired outer surface (skin) and skeleton (bone) meshes is available as a Biobank Returned Dataset. This research has been conducted using the UK Biobank Resource

GENTEL : GENerating Training data Efficiently for Learning to segment medical images

International audienceAccurately segmenting MRI images is crucial for many clinical applications. However, manually segmenting images with accurate pixel precision is a tedious and time consuming task. In this paper we present a simple, yet effective method to improve the efficiency of the image segmentation process. We propose to transform the image annotation task into a binary choice task. We start by using classical image processing algorithms with different parameter values to generate multiple, different segmentation masks for each input MRI image. Then, the user, instead of segmenting the pixels of the images, she/he only needs to decide if a segmentation is acceptable or not. This method allows us to efficiently obtain high quality segmentations with minor human intervention. With the selected segmentations we train a state-of-the-art neural network model. For the evaluation, we use a second MRI dataset (1.5T Dataset), acquired with a different protocol and containing annotations. We show that the trained network i) is capable to automatically segment cases where none of the classical methods obtained a high quality result ii) generalizes to the second MRI dataset, which was acquired with a different protocol and never seen at training time ; and iii) allows to detect miss-annotations in this second dataset. Quantitatively, the trained network obtains very good results : DICE score - mean 0.98, median 0.99- and Hausdorff distance (in pixels) - mean 4.7, median 2.0-.La segmentation précise d'images à résonnance magnétiques (IRM) est cruciale pour de nombreuses applications cliniques. Cependant, une segmentation manuelle visant une précision au niveau du pixel est une tâche longue et fastidieuse. Dans cet article, nous proposons une méthode simple pour améliorer l'efficacité de la segmentation d'images. Nous proposons de transformer la tâche d'annotation d'une image en une tâche de choix binaire. D'abord, nous utilisons plusieurs algorithmes classiques de traitement d'image pour générer plusieurs candidats de masques de segmentation. Ensuite, l'utilisat.eur.rice, au lieu de segmenter les pixels des images, décide si une segmentation est acceptable ou non. Cette méthode nous permet d'obtenir efficacement un grand nombre de segmentations de haute qualité avec une intervention humaine li-mitée. Avec les images et leurs segmentations sélectionnées, nous entrainons un réseau de neurones de l'état de l'art qui prédit les segmentations à partir des images d'entrée. Nous le validons sur un autre jeu de données IRM, acquis avec un protocole différent, et qui contient des segmentations. Nous montrons que le réseau entrainé 1) est capable de segmenter automatiquement des cas où aucune des méthodes classiques n'a obtenu un résultat de haute qualité, 2) est capable de segmenter un autre jeu de don-nées IRM, acquis avec un protocole différent et jamais vu lors de l'entrainement, et 3) permet de détecter des annotations erronées dans ce jeu de données. Quantitativement, le réseau entrainé obtient de très bons résultats : Score DICE-moyenne 0,98 et médiane 0,99-et distance de Hausdorff (en pixels)-moyenne 4,7, médiane 2,0

4DHumanOutfit: a multi-subject 4D dataset of human motion sequences in varying outfits exhibiting large displacements

This work presents 4DHumanOutfit, a new dataset of densely sampled spatio-temporal 4D human motion data of different actors, outfits and motions. The dataset is designed to contain different actors wearing different outfits while performing different motions in each outfit. In this way, the dataset can be seen as a cube of data containing 4D motion sequences along 3 axes with identity, outfit and motion. This rich dataset has numerous potential applications for the processing and creation of digital humans, e.g. augmented reality, avatar creation and virtual try on. 4DHumanOutfit is released for research purposes at https://kinovis.inria.fr/4dhumanoutfit/. In addition to image data and 4D reconstructions, the dataset includes reference solutions for each axis. We present independent baselines along each axis that demonstrate the value of these reference solutions for evaluation tasks

Analyse der Spontanmotorik im 1. Lebensjahr: Markerlose 3-D-Bewegungserfassung zur Früherkennung von Entwicklungsstörungen

Children with motor development disorders benefit greatly from early interventions. An early diagnosis in pediatric preventive care (U2–U5) can be improved by automated screening. Current approaches to automated motion analysis, however, are expensive, require lots of technical support, and cannot be used in broad clinical application. Here we present an inexpensive, marker-free video analysis tool (KineMAT) for infants, which digitizes 3‑D movements of the entire body over time allowing automated analysis in the future. Three-minute video sequences of spontaneously moving infants were recorded with a commercially available depth-imaging camera and aligned with a virtual infant body model (SMIL model). The virtual image generated allows any measurements to be carried out in 3‑D with high precision. We demonstrate seven infants with different diagnoses. A selection of possible movement parameters was quantified and aligned with diagnosis-specific movement characteristics. KineMAT and the SMIL model allow reliable, three-dimensional measurements of spontaneous activity in infants with a very low error rate. Based on machine-learning algorithms, KineMAT can be trained to automatically recognize pathological spontaneous motor skills. It is inexpensive and easy to use and can be developed into a screening tool for preventive care for children.Kinder mit motorischer Entwicklungsstörung profitieren von einer frühen Entwicklungsförderung. Eine frühe Diagnosestellung in der kinderärztlichen Vorsorge (U2–U5) kann durch ein automatisiertes Screening verbessert werden. Bisherige Ansätze einer automatisierten Bewegungsanalyse sind jedoch teuer und aufwendig und nicht in der Breite anwendbar. In diesem Beitrag soll ein neues System zur Videoanalyse, das Kinematic Motion Analysis Tool (KineMAT) vorgestellt werden. Es kann bei Säuglingen angewendet werden und kommt ohne Körpermarker aus. Die Methode wird anhand von 7 Patienten mit unterschiedlichen Diagnosen demonstriert. Mit einer kommerziell erhältlichen Tiefenbildkamera (RGB-D[Red-Green-Blue-Depth]-Kamera) werden 3‑minütige Videosequenzen von sich spontan bewegenden Säuglingen aufgenommen und mit einem virtuellen Säuglingskörpermodell (SMIL[Skinned Multi-infant Linear]-Modell) in Übereinstimmung gebracht. Das so erzeugte virtuelle Abbild erlaubt es, beliebige Messungen in 3‑D mit hoher Präzision durchzuführen. Eine Auswahl möglicher Bewegungsparameter wird mit diagnosespezifischen Bewegungsauffälligkeiten zusammengeführt. Der KineMAT und das SMIL-Modell erlauben eine zuverlässige, dreidimensionale Messung der Spontanaktivität bei Säuglingen mit einer sehr niedrigen Fehlerrate. Basierend auf maschinellen Lernalgorithmen kann der KineMAT trainiert werden, pathologische Spontanmotorik automatisiert zu erkennen. Er ist kostengünstig und einfach anzuwenden und soll als Screeninginstrument für die kinderärztliche Vorsorge weiterentwickelt werden

3DTeethSeg'22: 3D Teeth Scan Segmentation and Labeling Challenge

Teeth localization, segmentation, and labeling from intra-oral 3D scans are essential tasks in modern dentistry to enhance dental diagnostics, treatment planning, and population-based studies on oral health. However, developing automated algorithms for teeth analysis presents significant challenges due to variations in dental anatomy, imaging protocols, and limited availability of publicly accessible data. To address these challenges, the 3DTeethSeg'22 challenge was organized in conjunction with the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) in 2022, with a call for algorithms tackling teeth localization, segmentation, and labeling from intraoral 3D scans. A dataset comprising a total of 1800 scans from 900 patients was prepared, and each tooth was individually annotated by a human-machine hybrid algorithm. A total of 6 algorithms were evaluated on this dataset. In this study, we present the evaluation results of the 3DTeethSeg'22 challenge. The 3DTeethSeg'22 challenge code can be accessed at: https://github.com/abenhamadou/3DTeethSeg22_challengeComment: 29 pages, MICCAI 2022 Singapore, Satellite Event, Challeng