113 research outputs found
High-Performance and Tunable Stereo Reconstruction
Traditional stereo algorithms have focused their efforts on reconstruction
quality and have largely avoided prioritizing for run time performance. Robots,
on the other hand, require quick maneuverability and effective computation to
observe its immediate environment and perform tasks within it. In this work, we
propose a high-performance and tunable stereo disparity estimation method, with
a peak frame-rate of 120Hz (VGA resolution, on a single CPU-thread), that can
potentially enable robots to quickly reconstruct their immediate surroundings
and maneuver at high-speeds. Our key contribution is a disparity estimation
algorithm that iteratively approximates the scene depth via a piece-wise planar
mesh from stereo imagery, with a fast depth validation step for semi-dense
reconstruction. The mesh is initially seeded with sparsely matched keypoints,
and is recursively tessellated and refined as needed (via a resampling stage),
to provide the desired stereo disparity accuracy. The inherent simplicity and
speed of our approach, with the ability to tune it to a desired reconstruction
quality and runtime performance makes it a compelling solution for applications
in high-speed vehicles.Comment: Accepted to International Conference on Robotics and Automation
(ICRA) 2016; 8 pages, 5 figure
Learning Combinatorial Embedding Networks for Deep Graph Matching
Graph matching refers to finding node correspondence between graphs, such
that the corresponding node and edge's affinity can be maximized. In addition
with its NP-completeness nature, another important challenge is effective
modeling of the node-wise and structure-wise affinity across graphs and the
resulting objective, to guide the matching procedure effectively finding the
true matching against noises. To this end, this paper devises an end-to-end
differentiable deep network pipeline to learn the affinity for graph matching.
It involves a supervised permutation loss regarding with node correspondence to
capture the combinatorial nature for graph matching. Meanwhile deep graph
embedding models are adopted to parameterize both intra-graph and cross-graph
affinity functions, instead of the traditional shallow and simple parametric
forms e.g. a Gaussian kernel. The embedding can also effectively capture the
higher-order structure beyond second-order edges. The permutation loss model is
agnostic to the number of nodes, and the embedding model is shared among nodes
such that the network allows for varying numbers of nodes in graphs for
training and inference. Moreover, our network is class-agnostic with some
generalization capability across different categories. All these features are
welcomed for real-world applications. Experiments show its superiority against
state-of-the-art graph matching learning methods.Comment: ICCV2019 oral. Code available at
https://github.com/Thinklab-SJTU/PCA-G
Monocular 3D Scene Reconstruction for an Autonomous Unmanned Aerial Vehicle
Rekonstrukce 3D modelu prostĹ™edĂ je klĂÄŤovou částĂ autonomnĂho letu bezpilotnĂ helikoptĂ©ry (UAV). Kombinace inerciálnĂ měřicĂ jednotky (IMU) a kamery je běžnou a dostupnou senzorovou sadou, jeĹľ je schopna zĂskat informaci o měřĂtku prostĹ™edĂ. Tato práce si klade za cĂl vyvinout algoritmus Ĺ™ešĂcĂ problĂ©m 3D rekostrukce pro tyto senzory za vyuĹľitĂ existujĂcĂch metod vizuálnÄ›-inerciálnĂ lokalizace (VINS). V práci jsou navrĹľeny dva algoritmy, odlišenĂ© zpĹŻsobem, jakĂ˝m extrahujĂ korespondence mezi snĂmky: párovacĂ algoritmus se širokou bázĂ a algoritmus zaloĹľenĂ˝ na trackingu s malou bázĂ. TakĂ© je implementována metoda vylepšujĂcĂ vĂ˝slednou 3D strukturu po letu. Algoritmy jsou otestovány na veĹ™ejnÄ› dostupnĂ© datovĂ© sadÄ›. NavĂc jsou otestovány v simulátoru a je proveden experiment v reálnĂ©m prostĹ™edĂ. VĂ˝sledky ukazujĂ, Ĺľe algoritmus zaloĹľenĂ˝ na trackingu dosahuje vĂ˝raznÄ› lepšĂch vĂ˝sledkĹŻ. NavĂc testy na datech a experimenty v reálnĂ©m prostĹ™edĂ ukazujĂ, Ĺľe algoritmus mĹŻĹľe bĂ˝t nasazen v praktickĂ˝ch aplikaÄŤnĂch situacĂch.The real-time 3D reconstruction of the surrounding scene is a key part in the pipeline of the autonomous flight of unmanned aerial vehicle (UAV). The combination of an inertial measurement unit (IMU) and a monocular camera is a common and inexpensive sensor setup that can be used to recover the scale of the environment. This thesis aims to develop an algorithm solving this problem for this particular setup by leveraging the existing visual-inertial navigation system (VINS) odometry algorithms for localisation. Two algorithms are developed, wide-baseline matching-based and small-baseline tracking-based. Also, an offline post-processing structure-refinement step is implemented to further improve the resulting structure. The algorithms and the refinement step are then evaluated on publicly available datasets. Furthermore, they are tested in a simulator, and a real-world experiment is conducted. The results show that the tracking-based algorithm is significantly more performant. Importantly, tests on the datasets and the real-world experiments suggest that this algorithm can be practically employed in application scenarios
How to build a 2d and 3d aerial multispectral map?—all steps deeply explained
UIDB/04111/2020 PCIF/SSI/0102/2017 IF/00325/2015 UIDB/00066/2020The increased development of camera resolution, processing power, and aerial platforms helped to create more cost-efficient approaches to capture and generate point clouds to assist in scientific fields. The continuous development of methods to produce three-dimensional models based on two-dimensional images such as Structure from Motion (SfM) and Multi-View Stereopsis (MVS) allowed to improve the resolution of the produced models by a significant amount. By taking inspiration from the free and accessible workflow made available by OpenDroneMap, a detailed analysis of the processes is displayed in this paper. As of the writing of this paper, no literature was found that described in detail the necessary steps and processes that would allow the creation of digital models in two or three dimensions based on aerial images. With this, and based on the workflow of OpenDroneMap, a detailed study was performed. The digital model reconstruction process takes the initial aerial images obtained from the field survey and passes them through a series of stages. From each stage, a product is acquired and used for the following stage, for example, at the end of the initial stage a sparse reconstruction is produced, obtained by extracting features of the images and matching them, which is used in the following step, to increase its resolution. Additionally, from the analysis of the workflow, adaptations were made to the standard workflow in order to increase the compatibility of the developed system to different types of image sets. Particularly, adaptations focused on thermal imagery were made. Due to the low presence of strong features and therefore difficulty to match features across thermal images, a modification was implemented, so thermal models could be produced alongside the already implemented processes for multispectral and RGB image sets.publishersversionpublishe
Graph matching using position coordinates and local features for image analysis
Encontrar las correspondencias entre dos imágenes es un problema crucial en el campo de la visiĂłn por ordenador i el reconocimiento de patrones. Es relevante para un amplio rango de propĂłsitos des de aplicaciones de reconocimiento de objetos en las áreas de biometrĂa, análisis de documentos i análisis de formas hasta aplicaciones relacionadas con la geometrĂa desde mĂşltiples puntos de vista tales cĂłmo la recuperaciĂłn de la pose, estructura desde el movimiento y localizaciĂłn y mapeo.
La mayorĂa de las tĂ©cnicas existentes enfocan este problema o bien usando caracterĂsticas locales en la imagen o bien usando mĂ©todos de registro de conjuntos de puntos (o bien una mezcla de ambos). En las primeras, un conjunto disperso de caracterĂsticas es primeramente extraĂdo de las imágenes y luego caracterizado en la forma de vectores descriptores usando evidencias locales de la imagen. Las caracterĂsticas son asociadas segĂşn la similitud entre sus descriptores. En las segundas, los conjuntos de caracterĂsticas son considerados cĂłmo conjuntos de puntos los cuales son asociados usando tĂ©cnicas de optimizaciĂłn no lineal. Estos son procedimientos iterativos que estiman los parámetros de correspondencia y de alineamiento en pasos alternados.
Los grafos son representaciones que contemplan relaciones binarias entre las caracterĂsticas. Tener en cuenta relaciones binarias al problema de la correspondencia a menudo lleva al llamado problema del emparejamiento de grafos. Existe cierta cantidad de mĂ©todos en la literatura destinados a encontrar soluciones aproximadas a diferentes instancias del problema de emparejamiento de grafos, que en la mayorĂa de casos es del tipo "NP-hard".
El cuerpo de trabajo principal de esta tesis está dedicado a formular ambos problemas de asociaciĂłn de caracterĂsticas de imagen y registro de conjunto de puntos como instancias del problema de emparejamiento de grafos. En todos los casos proponemos algoritmos aproximados para solucionar estos problemas y nos comparamos con un nĂşmero de mĂ©todos existentes pertenecientes a diferentes áreas como eliminadores de "outliers", mĂ©todos de registro de conjuntos de puntos y otros mĂ©todos de emparejamiento de grafos.
Los experimentos muestran que en la mayorĂa de casos los mĂ©todos propuestos superan al resto. En ocasiones los mĂ©todos propuestos o bien comparten el mejor rendimiento con algĂşn mĂ©todo competidor o bien obtienen resultados ligeramente peores. En estos casos, los mĂ©todos propuestos normalmente presentan tiempos computacionales inferiores.Trobar les correspondències entre dues imatges Ă©s un problema crucial en el camp de la visiĂł per ordinador i el reconeixement de patrons. És rellevant per un ampli ventall de propòsits des d’aplicacions de reconeixement d’objectes en les Ă rees de biometria, anĂ lisi de documents i anĂ lisi de formes fins aplicacions relacionades amb geometria des de mĂşltiples punts de vista tals com recuperaciĂł de pose, estructura des del moviment i localitzaciĂł i mapeig.
La majoria de les tècniques existents enfoquen aquest problema o bĂ© usant caracterĂstiques locals a la imatge o bĂ© usant mètodes de registre de conjunts de punts (o bĂ© una mescla d’ambdĂłs). En les primeres, un conjunt dispers de caracterĂstiques Ă©s primerament extret de les imatges i desprĂ©s caracteritzat en la forma de vectors descriptors usant evidències locals de la imatge. Les caracterĂstiques son associades segons la similitud entre els seus descriptors. En les segones, els conjunts de caracterĂstiques son considerats com conjunts de punts els quals son associats usant tècniques d’optimitzaciĂł no lineal. Aquests son procediments iteratius que estimen els parĂ metres de correspondència i d’alineament en passos alternats.
Els grafs son representacions que contemplen relacions binaries entre les caracterĂstiques. Tenir en compte relacions binĂ ries al problema de la correspondència sovint porta a l’anomenat problema de l’emparellament de grafs. Existeix certa quantitat de mètodes a la literatura destinats a trobar solucions aproximades a diferents instĂ ncies del problema d’emparellament de grafs, el qual en la majoria de casos Ă©s del tipus “NP-hard”.
Una part del nostre treball estĂ dedicat a investigar els beneficis de les mesures de ``bins'' creuats per a la comparaciĂł de caracterĂstiques locals de les imatges.
La resta estĂ dedicat a formular ambdĂłs problemes d’associaciĂł de caracterĂstiques d’imatge i registre de conjunt de punts com a instĂ ncies del problema d’emparellament de grafs. En tots els casos proposem algoritmes aproximats per solucionar aquests problemes i ens comparem amb un nombre de mètodes existents pertanyents a diferents Ă rees com eliminadors d’“outliers”, mètodes de registre de conjunts de punts i altres mètodes d’emparellament de grafs.
Els experiments mostren que en la majoria de casos els mètodes proposats superen a la resta. En ocasions els mètodes proposats o bé comparteixen el millor rendiment amb algun mètode competidor o bé obtenen resultats lleugerament pitjors. En aquests casos, els mètodes proposats normalment presenten temps computacionals inferiors
A New Computational Framework for Efficient Parallelization and Optimization of Large Scale Graph Matching
There are so many applications in data fusion, comparison, and recognition that require a robust and efficient algorithm to match features of multiple images. To improve accuracy and get a more stable result is important to take into consideration both local appearance and the pairwise relationship of features. Graphs are a powerful and flexible data structure, allowing for the description of complex relationships between data elements, whose nodes correspond to salient features and edges correspond to relational aspects between features. Therefore, the problem of graph matching is to find a mapping between the two sets of nodes that preserves the relationships between them as much as possible. This graph-matching problem is mathematically formulated as an IQP problem which solving it is NP-hard, and obtaining exact Optima only plausible for very small data. Therefore, handling large-scale scientific visual data is quite limited, necessitating both efficient serial algorithms, as well as scalable parallel formulations. In this thesis, we first focused on exploring techniques to reduce the computation cost as well as memory usage of Pairwise graph matching by adopting a heuristic pruning strategy together with a redundancy pattern suppression scheme. We also modified the structure of the affinity matrix for minimizing memory requirement and parallelizing our algorithm by employing CPU’s and GPU’s accelerated libraries. Any pair of features with similar distance from first image results in same sub-matrices, therefore instead of constructing the whole affinity matrix, we only built the sub-blocked affinity for those distinct feature distances. By employing this scheme not only saved large memory and reduced computation time tremendously but also, the matrix-vector multiplication of gradient computation performed in parallel, where each block-vector calculation computed independently without synchronization. The accelerated libraries such as MKL, cuSparse, cuBlas and thrust applied to solving the GM problem, following the scheme of the spectral matching algorithm. We also extended our work for Multi-graph imaging, since many tasks require finding correspondences across multiple images. Also, considering more graph improves the matching accuracy. Most algorithms obtain approximate solutions for solving the GM NP-hard problem, result in a weak optimal solution. Therefore, we proposed a new solver, which iteratively modified the affinity matrix and binarized the solution by optimizing the original problem with its integer constraints
- …