Unordered feature tracking made fast and easy

International audienceWe present an efficient algorithm to fuse two-view correspondences into multi-view consistent tracks. The proposed method relies on the Union-Find algorithm to solve the fusion problem. It is very simple and has a lower computational complexity than other available methods. Our experiments show that it is faster and computes more tracks

Python Photogrammetry Toolbox: A free solution for Three-Dimensional Documentation

International audienceThe modern techniques of Structure from Motion (SfM) and Image-Based Modelling (IBM) open new perspectives in the field of archaeological documentation, providing a simple and accurate way to record three-dimensional data. In the last edition of the workshop, the presentation "Computer Vision and Structure From Motion, new methodologies in archaeological three-dimensional documentation. An open source approach." showed the advantages of this new methodology (low cost, portability, versatility ...), but it also identified some problems: the use of the closed feature detector SIFT source code and the necessity of a simplification of the workflow. The software Python Photogrammetry Toolbox (PPT) is a possible solution to solve these problems. It is composed of python scripts that automate the different steps of the workflow. The entire process is reduced in two commands, calibration and dense reconstruction. The user can run it from a graphical interface or from terminal command

To 3D or Not 3D: Choosing a Photogrammetry Workflow for Cultural Heritage Groups

The 3D reconstruction of real-world heritage objects using either a laser scanner or 3D modelling software is typically expensive and requires a high level of expertise. Image-based 3D modelling software, on the other hand, offers a cheaper alternative, which can handle this task with relative ease. There also exists free and open source (FOSS) software, with the potential to deliver quality data for heritage documentation purposes. However, contemporary academic discourse seldom presents survey-based feature lists or a critical inspection of potential production pipelines, nor typically provides direction and guidance for non-experts who are interested in learning, developing and sharing 3D content on a restricted budget. To address the above issues, a set of FOSS were studied based on their offered features, workflow, 3D processing time and accuracy. Two datasets have been used to compare and evaluate the FOSS applications based on the point clouds they produced. The average deviation to ground truth data produced by a commercial software application (Metashape, formerly called PhotoScan) was used and measured with CloudCompare software. 3D reconstructions generated from FOSS produce promising results, with significant accuracy, and are easy to use. We believe this investigation will help non-expert users to understand the photogrammetry and select the most suitable software for producing image-based 3D models at low cost for visualisation and presentation purposes

Estimation robuste de modèle a contrario, impact sur la précision en structure from motion

L'estimation de modèle consiste à identifier un modèle parmi des données bruitées. Ce problème n'est pas trivial et l'état de l'art présente de nombreuses solutions pour résoudre ce problème. Le plus souvent les solutions max-consensus ou RANSAC sont utilisées. Ces solutions proposent de rechercher par tirages aléatoires plusieurs solutions et de conserver celle qui présente le plus grand cardinal face à une précision donnée a priori. Ces solutions, malgrè leur simplicité, présentent un défaut majeur : un seuil d'acception des données T doit être spécifié. Il se pose alors la question du choix de ce paramètre. Choisir un seuil trop grand va donner lieu à une sur-estimation des données valides et l'on va introduire des données bruitées dans le modèle alors que choisir un seuil trop petit donne lieu à une sous-estimation et une imprécision du modèle. Nous proposons de discuter la solution AC-RANSAC pour le Structure from Motion et son impact sur la précision des positions de caméras estimées

Positionnement robuste et précis de réseaux d’images

To compute a 3D representation of a rigid scene from a collection of pictures is now possible thanks to the progress made by the multiple-view stereovision methods, even with a simple camera. The reconstruction process, arising from photogrammetry, consists in integrating information from multiple images taken from different viewpoints in order to identify the relative positions and orientations. Once the positions and orientations (external calibration) of the cameras are retrieved, the structure of the scene can be reconstructed. To solve the problem of calculating the Structure from Motion (SfM), sequential and global methods have been proposed. By nature, sequential methods tend to accumulate errors. This is observable in trajectories of cameras that are subject to drift error. When pictures are acquired around an object it leads to reconstructions where the loops do not close. In contrast, global methods consider the network of cameras as a whole. The configuration of cameras is searched and optimized in order to preserve at best the constraints of the cyclical network. Reconstructions of better quality can be obtained, but at the expense of computation time. This thesis aims at analyzing critical issues at the heart of these methods of external calibration and at providing solutions to improve their performance(accuracy , robustness and speed) and their ease of use (restricted parametrization).We first propose a fast and efficient feature tracking algorithm. We then show that the widespread use of a contrario robust estimation of parametric models frees the user from choosing detection thresholds, and allows obtaining a reconstruction pipeline that automatically adapts to the data. Then in a second step, we use the adaptive robust estimation and a series of convex optimizations to build a scalable global calibration chain. Our experiments show that the a contrario based estimations improve significantly the quality of the pictures positions and orientations, while being automatic and without parameters, even on complex camera networks. Finally, we propose to improve the visual appearance of the reconstruction by providing a convex optimization to ensure the color consistency between imagesCalculer une représentation 3D d'une scène rigide à partir d'une collection d'images est aujourd'hui possible grâce aux progrès réalisés par les méthodes de stéréo-vision multi-vues, et ce avec un simple appareil photographique. Le principe de reconstruction, découlant de travaux de photogrammétrie, consiste à recouper les informations provenant de plusieurs images, prises de points de vue différents, pour identifier les positions et orientations relatives de chaque cliché. Une fois les positions et orientations de caméras déterminées (calibration externe), la structure de la scène peut être reconstruite. Afin de résoudre le problème de calcul de la structure à partir du mouvement des caméras (Structure-from-Motion), des méthodes séquentielles et globales ont été proposées. Par nature, les méthodes séquentielles ont tendance à accumuler les erreurs. Cela donne lieu le plus souvent à des trajectoires de caméras qui dérivent et, lorsque les photos sont acquises autour d'un objet, à des reconstructions où les boucles ne se referment pas. Au contraire, les méthodes globales considèrent le réseau de caméras dans son ensemble. La configuration de caméras est recherchée et optimisée pour conserver au mieux l'ensemble des contraintes de cyclicité du réseau. Des reconstructions de meilleure qualité peuvent être obtenues, au détriment toutefois du temps de calcul. Cette thèse propose d'analyser des problèmes critiques au cœur de ces méthodes de calibration externe et de fournir des solutions pour améliorer leur performance (précision, robustesse, vitesse) et leur facilité d'utilisation (paramétrisation restreinte).Nous proposons tout d'abord un algorithme de suivi de points rapide et efficace. Nous montrons ensuite que l'utilisation généralisée de l'estimation robuste de modèles paramétriques a contrario permet de libérer l'utilisateur du réglage de seuils de détection, et d'obtenir une chaine de reconstruction qui s'adapte automatiquement aux données. Puis dans un second temps, nous utilisons ces estimations robustes adaptatives et une formulation du problème qui permet des optimisations convexes pour construire une chaine de calibration globale capable de passer à l'échelle. Nos expériences démontrent que les estimations identifiées a contrario améliorent de manière notable la qualité d'estimation de la position et de l'orientation des clichés, tout en étant automatiques et sans paramètres, et ce même sur des réseaux de caméras complexes. Nous proposons enfin d'améliorer le rendu visuel des reconstructions en proposant une optimisation convexe de la consistance colorée entre image

EMEA and Gene Therapy Medicinal Products Development in the European Union

The evaluation of quality, safety, and efficacy of medicinal products by the European Medicines Evaluation Agency (EMEA) via the centralized procedure is the only available regulatory procedure for obtaining marketing authorization for gene therapy (GT) medicinal products in the European Union. The responsibility for the authorization of clinical trials remains with the national competent authorities (NCA) acting in a harmonized framework from the scientific viewpoint. With the entry into force of a new directive on good clinical practice implementation in clinical trials as of 1 May 2004, procedural aspects will also be harmonized at EU level. Scientifically sound development of medicinal products is the key for the successful registration of dossiers and for contributing to the promotion and protection of public health. The objective of this paper is to introduce the EMEA regulatory processes and scientific activities relevant to GT medicinal products

Chat2Map: Efficient Scene Mapping from Multi-Ego Conversations

Can conversational videos captured from multiple egocentric viewpoints reveal the map of a scene in a cost-efficient way? We seek to answer this question by proposing a new problem: efficiently building the map of a previously unseen 3D environment by exploiting shared information in the egocentric audio-visual observations of participants in a natural conversation. Our hypothesis is that as multiple people ("egos") move in a scene and talk among themselves, they receive rich audio-visual cues that can help uncover the unseen areas of the scene. Given the high cost of continuously processing egocentric visual streams, we further explore how to actively coordinate the sampling of visual information, so as to minimize redundancy and reduce power use. To that end, we present an audio-visual deep reinforcement learning approach that works with our shared scene mapper to selectively turn on the camera to efficiently chart out the space. We evaluate the approach using a state-of-the-art audio-visual simulator for 3D scenes as well as real-world video. Our model outperforms previous state-of-the-art mapping methods, and achieves an excellent cost-accuracy tradeoff. Project: http://vision.cs.utexas.edu/projects/chat2map.Comment: Accepted to CVPR 202

A New Type of Metal Recognition by Human T Cells: Contact Residues for Peptide-independent Bridging of T Cell Receptor and Major Histocompatibility Complex by Nickel

In spite of high frequencies of metal allergies, the structural basis for major histocompatibility complex (MHC)-restricted metal recognition is among the unanswered questions in the field of T cell activation. For the human T cell clone SE9, we have identified potential Ni contact sites in the T cell receptor (TCR) and the restricting human histocompatibility leukocyte antigen (HLA)-DR structure. The specificity of this HLA-DR–promiscuous VA22/VB17+ TCR is primarily harbored in its α chain. Ni reactivity is neither dependent on protein processing in antigen-presenting cells nor affected by the nature of HLA-DR–associated peptides. However, SE9 activation by Ni crucially depends on Tyr29 in CDR1α, an N-nucleotide–encoded Tyr94 in CDR3α, and a conserved His81 in the HLA-DR β chain. These data indicate that labile, nonactivating complexes between the SE9 TCR and most HLA-DR/peptide conjugates might supply sterically optimized coordination sites for Ni ions, three of which were identified in this study. In such complexes Ni may effectively bridge the TCR α chain to His81 of most DR molecules. Thus, in analogy to superantigens, Ni may directly link TCR and MHC in a peptide-independent manner. However, unlike superantigens, Ni requires idiotypic, i.e., CDR3α-determined TCR amino acids. This new type of TCR–MHC linkage might explain the high frequency of Ni-reactive T cells in the human population