Study of the brain connectivity in an Immersive Space

National audienceVirtual reality is a powerful tool for scientific visualization. When the amount and complexity of the visualized data grows, standard visualization applications on desktop computers become inefficient. In this paper we present the use of a CAVE like VR facility in a neuroscientific context. The aim is to have a better understanding of the brain connectivity. Both anatomical and functional data are attached to a mesh representing the brain surface. Specific tools developed for this study and the way we used them are presented below, emphasizing drawbacks and advantages of virtual reality in a scientific visualization context

Bi-manual 3D Painting: an interaction paradigm for augmented reality live performance

International audienceThe rise of gestural interaction led artists to produce shows, or installations based on this paradigm. We present the first stages of the " Sculpture numérique " (Virtual Sculpture) project. This project was born from a collaboration with dancers. Its goal is to propose bi-manual interactions in a large augmented space: we aim at giving dancers the possibility to generate and manipulate virtual elements on stage using their hands. The first set of interactions we present in this paper is 3D painting, where the user can generate 3D virtual matter from his hands. The movement of the hand defines a stroke, which shape is controlled by the shape of the hand. Changing the shape and orientation of the hand allows switching between three interaction modes to produce volumes, surfaces or curves in space. We explore the applicative case of dance, with the goal of producing a plastic creation from choreography.La généralisation au grand public de l'interaction gestuelle et de la réalité augmentée a amené les artistes à s'approprier ces technologies et à les intégrer dans la production de spectacles ou d'installations. Dans cet article, nous présentons les premiers résultats du projet "Sculpture Numérique". Issu d'une collaboration avec le milieu de la danse, ce projet a pour but de contribuer sur l'interaction bi-manuelle dans un grand espace augmenté. La première étape que nous proposons ici est la Peinture 3D: la génération de matière virtuelle à la main dans un espace large. Par son mouvement, la main de l'utilisateur dessine dans l'espace 3D un trait, une surface ou un volume défini par la conformation de sa main et par son mouvement. Celui-ci peut ainsi dessiner des formes dans l'espace. Cette technique, généralisable à d'autres domaines, est ici appliquée à la danse, dans le but de permettre à un danseur ou chorégraphe de produire une création plastique virtuelle issue d'une chorégraphie

Unsupervised Individual Whales Identification: Spot the Difference in the Ocean

International audienceIdentifying organisms is a key step in accessing information related to the ecology of species. But unfortunately, this is difficult to achieve due to the level of expertise necessary to correctly identify and record living organisms. To try bridging this gap, enormous work has been done on the development of automated species identification tools such as image-based plant identification or audio recordings-based bird identification. Yet, for some groups, it is preferable to monitor the organisms at the individual level rather than at the species level. The automatizing of this problem has received much less attention than species identification. In this paper, we address the specific scenario of discovering humpack whales individuals in a large collections of pictures collected by nature observers. The process is initiated from scratch, without any knowledge on the number of individuals and without any training samples of these individuals. Thus, the problem is entirely unsupervised. To address it, we set up and experimented a scalable fine-grained matching system allowing to discover small rigid visual patterns in highly clutter background. The evaluation was conducted in blind in the context of the LifeCLEF evaluation campaign. Results show that the proposed system provides very promising results with regard to the difficulty of the task but that there is still room for improvements to reach higher recall and precision in the future

Improving realism of a surgery simulator: linear anisotropic elasticity, complex interactions and force extrapolation

International audienceIn this article, we describe the latest developments of the minimally invasive hepatic surgery simulator prototype developed at INRIA. The goal of this simulator is to provide a realistic training test bed to perform laparoscopic procedures. Therefore, its main functionality is to simulate the action of virtual laparoscopic surgical instruments for deforming and cutting tridimensional anatomical models. Throughout this paper, we present the general features of this simulator including the implementation of several biomechanical models and the integration of two force-feedback devices in the simulation platform. More precisely, we describe three new important developments that improve the overall realism of our simulator. First, we have developed biomechanical models, based on linear elasticity and finite element theory, that include the notion of anisotropic deformation. Indeed, we have generalized the linear elastic behaviour of anatomical models to 'transversally isotropic' materials, i.e. materials having a different behaviour in a given direction. We have also added to the volumetric model an external elastic membrane representing the 'liver capsule', a rather stiff skin surrounding the liver, which creates a kind of 'surface anisotropy'. Second, we have developed new contact models between surgical instruments and soft tissue models. For instance, after detecting a contact with an instrument, we define specific boundary constraints on deformable models to represent various forms of interactions with a surgical tool, such as sliding, gripping, cutting or burning. In addition, we compute the reaction forces that should be felt by the user manipulating the force-feedback devices. The last improvement is related to the problem of haptic rendering. Currently, we are able to achieve a simulation frequency of 25 Hz (visual real time) with anatomical models of complex geometry and behaviour. But to achieve a good haptic feedback requires a frequency update of applied forces typically above 300 Hz (haptic real time). Thus, we propose a force extrapolation algorithm in order to reach haptic real time

Improving Realism of a Surgery Simulator : Linear Anisotropic Elasticity, Complex Interactions and Force Extrapolation

In this article, we describe the latest developments of the minimally invasive hepatic surgery simulator prototype developed at INRIA. The goal of this simulator is to provide a realistic training test bed for performing laparoscopic procedures. Therefore, its main functionality is to simulate the action of virtual laparoscopic surgical instruments for deforming and cutting tridimensional anatomical models. Throughout this paper, we present the general features of this simulator including the implementation of several biomechanical models and the integration of two force-feedback devices in the simulation platform. More precisely, we describe three new important developments that contribute to improve the overall realism of our simulator. First, we have developed bio-mechanical models, based on linear elasticity and finite element theory, that include the notion of anisotropic deformation. Indeed, we have generalized the linear elastic behavior of anatomical models to "transversally isotropic" materials, i.e. materials having a different behavior in one given direction. We have also added to the volumetric model an external elastic membrane standing for the "liver capsule", a quite stiff skin surrounding the liver and creating a kind of "surface anisotropy". Second, we have developed new contact models between surgical instruments and soft tissue models. For instance, after detecting a contact with an instrument, we define specific boundary constraints on deformable models to represent various forms of interactions with a surgical tool, such as sliding, gripping, cutting or burning. In addition, we compute the reaction forces that should be felt by the user manipulating the force-feedback devices. The last improvement is related to the problem of haptic rendering. Currently , we are able to achieve a simulation frequency of 25Hz (visual real-time) with anatomical models of complex geometry and behavior. But to achieve a good haptic feedback requires a frequency update of applied forces typically above 300Hz (haptic real-time). Thus, we propose a force extrapolat- ion algorithm in order to reach haptic real-time

Peinture numérique 3D bi-manuelle: un paradigme d'interaction pour la réalité augmentée en spectacle vivant

National audienceThe rise of gestural interaction led artists to produce shows, or installations based on this paradigm. We present the first stages of the " Sculpture numérique " (Virtual Sculpture) project. This project was born from a collaboration with dancers. Its goal is to propose bi-manual interactions in a large augmented space: we aim at giving dancers the possibility to generate and manipulate virtual elements on stage using their hands. The first set of interactions we present in this paper is 3D painting, where the user can generate 3D virtual matter from his hands. The movement of the hand defines a stroke, which shape is controlled by the shape of the hand. Changing the shape and orientation of the hand allows switching between three interaction modes to produce volumes, surfaces or curves in space. We explore the applicative case of dance, with the goal of producing a plastic creation from choreography

Interactive and Immersive Simulation using the Continuous Sensitivity Equation Method with Application to Thermal Transport

The objective of this work is to develop a computational method that allows the user to visualize the solution of a PDE system (Partial Differential Equations) in virtual reality facilities and interact with the computation at real time. This could be possible only if a fast estimate of the physical solution is constructed. In the present work, we propose to employ a first-order Taylor series expansion to extrapolate the solution field from a reference solution, as some parameters vary. The continuous sensitivity equation method is used to compute the sensitivity field for the reference solution. The methodology proposed is applied to a heat transport problem (advection - diffusion) and demonstrated in a virtual reality facility.L'objectif de ce travail est le développement d'une méthode de calcul permettant à un utilisateur de visualiser la solution d'un système d'EDP (Equations aux Dérivées Partielles) dans une salle de réalité virtuelle et d'interagir avec le calcul en temps réel. Ceci ne peut être possible que si une estimation rapide de la solution physique est construite. Dans ce travail, on propose d'employer une série de Taylor au premier ordre pour extrapoler le champ solution à partir d'une solution de référence lorsque des paramètres varient. La méthode de l'équation des sensibilités continues est utilisée pour calculer le champ de sensibilité pour la solution de référence. La méthodologie proposée est appliquée à un problème de transport thermique (advection - diffusion) et exposée dans une salle de réalité virtuelle

Pl@ntNet-300K: a plant image dataset with high label ambiguity and a long-tailed distribution

International audienceThis paper presents a novel image dataset with high intrinsic ambiguity and a longtailed distribution built from the database of Pl@ntNet citizen observatory. It consists of 306,146 plant images covering 1,081 species. We highlight two particular features of the dataset, inherent to the way the images are acquired and to the intrinsic diversity of plants morphology: (i) the dataset has a strong class imbalance, i.e., a few species account for most of the images, and, (ii) many species are visually similar, rendering identification difficult even for the expert eye. These two characteristics make the present dataset well suited for the evaluation of set-valued classification methods and algorithms. Therefore, we recommend two set-valued evaluation metrics associated with the dataset (macro-average top-k accuracy and macro-average average-k accuracy) and we provide baseline results established by training deep neural networks using the cross-entropy loss

Curved sensors: experimental performance of CMOS prototypes and wide field related imagers

The emergence of curved sensors technologies opens a new way to design compact high-performance optical systems. Recent progress on the French activity on curved sensors are presented in terms of optical performance and experimental results. The existing prototypes are demonstrated at TRL4, for VIS and SWIR domains. We present the roadmap jointly developed by CEA and CNRS to reach a higher TRL either on the performance of the devices or on the mass production processes. We present the results obtained on two demonstrators