単眼画像入力型ビューベース3Dモデル検索に関する研究

Tohoku University北村喜文課

ImMesh: An Immediate LiDAR Localization and Meshing Framework

In this paper, we propose a novel LiDAR(-inertial) odometry and mapping framework to achieve the goal of simultaneous localization and meshing in real-time. This proposed framework termed ImMesh comprises four tightly-coupled modules: receiver, localization, meshing, and broadcaster. The localization module utilizes the prepossessed sensor data from the receiver, estimates the sensor pose online by registering LiDAR scans to maps, and dynamically grows the map. Then, our meshing module takes the registered LiDAR scan for incrementally reconstructing the triangle mesh on the fly. Finally, the real-time odometry, map, and mesh are published via our broadcaster. The key contribution of this work is the meshing module, which represents a scene by an efficient hierarchical voxels structure, performs fast finding of voxels observed by new scans, and reconstructs triangle facets in each voxel in an incremental manner. This voxel-wise meshing operation is delicately designed for the purpose of efficiency; it first performs a dimension reduction by projecting 3D points to a 2D local plane contained in the voxel, and then executes the meshing operation with pull, commit and push steps for incremental reconstruction of triangle facets. To the best of our knowledge, this is the first work in literature that can reconstruct online the triangle mesh of large-scale scenes, just relying on a standard CPU without GPU acceleration. To share our findings and make contributions to the community, we make our code publicly available on our GitHub: https://github.com/hku-mars/ImMesh

Near-Optimal Motion Planning Algorithms Via A Topological and Geometric Perspective

Motion planning is a fundamental problem in robotics, which involves finding a path for an autonomous system, such as a robot, from a given source to a destination while avoiding collisions with obstacles. The properties of the planning space heavily influence the performance of existing motion planning algorithms, which can pose significant challenges in handling complex regions, such as narrow passages or cluttered environments, even for simple objects. The problem of motion planning becomes deterministic if the details of the space are fully known, which is often difficult to achieve in constantly changing environments. Sampling-based algorithms are widely used among motion planning paradigms because they capture the topology of space into a roadmap. These planners have successfully solved high-dimensional planning problems with a probabilistic-complete guarantee, i.e., it guarantees to find a path if one exists as the number of vertices goes to infinity. Despite their progress, these methods have failed to optimize the sub-region information of the environment for reuse by other planners. This results in re-planning overhead at each execution, affecting the performance complexity for computation time and memory space usage. In this research, we address the problem by focusing on the theoretical foundation of the algorithmic approach that leverages the strengths of sampling-based motion planners and the Topological Data Analysis methods to extract intricate properties of the environment. The work contributes a novel algorithm to overcome the performance shortcomings of existing motion planners by capturing and preserving the essential topological and geometric features to generate a homotopy-equivalent roadmap of the environment. This roadmap provides a mathematically rich representation of the environment, including an approximate measure of the collision-free space. In addition, the roadmap graph vertices sampled close to the obstacles exhibit advantages when navigating through narrow passages and cluttered environments, making obstacle-avoidance path planning significantly more efficient. The application of the proposed algorithms solves motion planning problems, such as sub-optimal planning, diverse path planning, and fault-tolerant planning, by demonstrating the improvement in computational performance and path quality. Furthermore, we explore the potential of these algorithms in solving computational biology problems, particularly in finding optimal binding positions for protein-ligand or protein-protein interactions. Overall, our work contributes a new way to classify routes in higher dimensional space and shows promising results for high-dimensional robots, such as articulated linkage robots. The findings of this research provide a comprehensive solution to motion planning problems and offer a new perspective on solving computational biology problems

A Survey of Surface Reconstruction from Point Clouds

International audienceThe area of surface reconstruction has seen substantial progress in the past two decades. The traditional problem addressed by surface reconstruction is to recover the digital representation of a physical shape that has been scanned, where the scanned data contains a wide variety of defects. While much of the earlier work has been focused on reconstructing a piece-wise smooth representation of the original shape, recent work has taken on more specialized priors to address significantly challenging data imperfections, where the reconstruction can take on different representations – not necessarily the explicit geometry. We survey the field of surface reconstruction, and provide a categorization with respect to priors, data imperfections, and reconstruction output. By considering a holistic view of surface reconstruction, we show a detailed characterization of the field, highlight similarities between diverse reconstruction techniques, and provide directions for future work in surface reconstruction

Contributions to the Content-Based Image Retrieval Using Pictorial Queris

L'accés massiu a les càmeres digitals, els ordinadors personals i a Internet, ha propiciat la creació de grans volums de dades en format digital. En aquest context, cada vegada adquireixen major rellevància totes aquelles eines dissenyades per organitzar la informació i facilitar la seva cerca.Les imatges són un cas particular de dades que requereixen tècniques específiques de descripció i indexació. L'àrea de la visió per computador encarregada de l'estudi d'aquestes tècniques rep el nom de Recuperació d'Imatges per Contingut, en anglès Content-Based Image Retrieval (CBIR). Els sistemes de CBIR no utilitzen descripcions basades en text sinó que es basen en característiques extretes de les pròpies imatges. En contrast a les més de 6000 llengües parlades en el món, les descripcions basades en característiques visuals representen una via d'expressió universal.La intensa recerca en el camp dels sistemes de CBIR s'ha aplicat en àrees de coneixement molt diverses. Així doncs s'han desenvolupat aplicacions de CBIR relacionades amb la medicina, la protecció de la propietat intel·lectual, el periodisme, el disseny gràfic, la cerca d'informació en Internet, la preservació dels patrimoni cultural, etc. Un dels punts importants d'una aplicació de CBIR resideix en el disseny de les funcions de l'usuari. L'usuari és l'encarregat de formular les consultes a partir de les quals es fa la cerca de les imatges. Nosaltres hem centrat l'atenció en aquells sistemes en què la consulta es formula a partir d'una representació pictòrica. Hem plantejat una taxonomia dels sistemes de consulta en composada per quatre paradigmes diferents: Consulta-segons-Selecció, Consulta-segons-Composició-Icònica, Consulta-segons-Esboç i Consulta-segons-Il·lustració. Cada paradigma incorpora un nivell diferent en el potencial expressiu de l'usuari. Des de la simple selecció d'una imatge, fins a la creació d'una il·lustració en color, l'usuari és qui pren el control de les dades d'entrada del sistema. Al llarg dels capítols d'aquesta tesi hem analitzat la influència que cada paradigma de consulta exerceix en els processos interns d'un sistema de CBIR. D'aquesta manera també hem proposat un conjunt de contribucions que hem exemplificat des d'un punt de vista pràctic mitjançant una aplicació final

Contributions to the content-based image retrieval using pictorial queries

Descripció del recurs: el 02 de novembre de 2010L'accés massiu a les càmeres digitals, els ordinadors personals i a Internet, ha propiciat la creació de grans volums de dades en format digital. En aquest context, cada vegada adquireixen major rellevància totes aquelles eines dissenyades per organitzar la informació i facilitar la seva cerca. Les imatges són un cas particular de dades que requereixen tècniques específiques de descripció i indexació. L'àrea de la visió per computador encarregada de l'estudi d'aquestes tècniques rep el nom de Recuperació d'Imatges per Contingut, en anglès Content-Based Image Retrieval (CBIR). Els sistemes de CBIR no utilitzen descripcions basades en text sinó que es basen en característiques extretes de les pròpies imatges. En contrast a les més de 6000 llengües parlades en el món, les descripcions basades en característiques visuals representen una via d'expressió universal. La intensa recerca en el camp dels sistemes de CBIR s'ha aplicat en àrees de coneixement molt diverses. Així doncs s'han desenvolupat aplicacions de CBIR relacionades amb la medicina, la protecció de la propietat intel·lectual, el periodisme, el disseny gràfic, la cerca d'informació en Internet, la preservació dels patrimoni cultural, etc. Un dels punts importants d'una aplicació de CBIR resideix en el disseny de les funcions de l'usuari. L'usuari és l'encarregat de formular les consultes a partir de les quals es fa la cerca de les imatges. Nosaltres hem centrat l'atenció en aquells sistemes en què la consulta es formula a partir d'una representació pictòrica. Hem plantejat una taxonomia dels sistemes de consulta en composada per quatre paradigmes diferents: Consulta-segons-Selecció, Consulta-segons-Composició-Icònica, Consulta-segons-Esboç i Consulta-segons-Il·lustració. Cada paradigma incorpora un nivell diferent en el potencial expressiu de l'usuari. Des de la simple selecció d'una imatge, fins a la creació d'una il·lustració en color, l'usuari és qui pren el control de les dades d'entrada del sistema. Al llarg dels capítols d'aquesta tesi hem analitzat la influència que cada paradigma de consulta exerceix en els processos interns d'un sistema de CBIR. D'aquesta manera també hem proposat un conjunt de contribucions que hem exemplificat des d'un punt de vista pràctic mitjançant una aplicació final

Automated Pattern Detection and Generalization of Building Groups

This dissertation focuses on the topic of building group generalization by considering the detection of building patterns. Generalization is an important research field in cartography, which is part of map production and the basis for the derivation of multiple representation. As one of the most important features on map, buildings occupy large amount of map space and normally have complex shape and spatial distribution, which leads to that the generalization of buildings has long been an important and challenging task. For social, architectural and geographical reasons, the buildings were built with some special rules which forms different building patterns. Building patterns are crucial structures which should be carefully considered during graphical representation and generalization. Although people can effortlessly perceive these patterns, however, building patterns are not explicitly described in building datasets. Therefore, to better support the subsequent generalization process, it is important to automatically recognize building patterns. The objective of this dissertation is to develop effective methods to detect building patterns from building groups. Based on the identified patterns, some generalization methods are proposed to fulfill the task of building generalization. The main contribution of the dissertation is described as the following five aspects: (1) The terminology and concept of building pattern has been clearly explained; a detailed and relative complete typology of building patterns has been proposed by summarizing the previous researches as well as extending by the author; (2) A stroke-mesh based method has been developed to group buildings and detect different patterns from the building groups; (3) Through the analogy between line simplification and linear building group typification, a stroke simplification based typification method has been developed aiming at solving the generalization of building groups with linear patterns; (4) A mesh-based typification method has been developed for the generalization of the building groups with grid patterns; (5) A method of extracting hierarchical skeleton structures from discrete buildings have been proposed. The extracted hierarchical skeleton structures are regarded as the representations of the global shape of the entire region, which is used to control the generalization process. With the above methods, the building patterns are detected from the building groups and the generalization of building groups are executed based on the patterns. In addition, the thesis has also discussed the drawbacks of the methods and gave the potential solutions.:Abstract I Kurzfassung III Contents V List of Figures IX List of Tables XIII List of Abbreviations XIV Chapter 1 Introduction 1 1.1 Background and motivation 1 1.1.1 Cartographic generalization 1 1.1.2 Urban building and building patterns 1 1.1.3 Building generalization 3 1.1.4 Hierarchical property in geographical objects 3 1.2 Research objectives 4 1.3 Study area 5 1.4 Thesis structure 6 Chapter 2 State of the Art 8 2.1 Operators for building generalization 8 2.1.1 Selection 9 2.1.2 Aggregation 9 2.1.3 Simplification 10 2.1.4 Displacement 10 2.2 Researches of building grouping and pattern detection 11 2.2.1 Building grouping 11 2.2.2 Pattern detection 12 2.2.3 Problem analysis . 14 2.3 Researches of building typification 14 2.3.1 Global typification 15 2.3.2 Local typification 15 2.3.3 Comparison analysis 16 2.3.4 Problem analysis 17 2.4 Summary 17 Chapter 3 Using stroke and mesh to recognize building group patterns 18 3.1 Abstract 19 3.2 Introduction 19 3.3 Literature review 20 3.4 Building pattern typology and study area 22 3.4.1 Building pattern typology 22 3.4.2 Study area 24 3.5 Methodology 25 3.5.1 Generating and refining proximity graph 25 3.5.2 Generating stroke and mesh 29 3.5.3 Building pattern recognition 31 3.6 Experiments 33 3.6.1 Data derivation and test framework 33 3.6.2 Pattern recognition results 35 3.6.3 Evaluation 39 3.7 Discussion 40 3.7.1 Adaptation of parameters 40 3.7.2 Ambiguity of building patterns 44 3.7.3 Advantage and Limitation 45 3.8 Conclusion 46 Chapter 4 A typification method for linear building groups based on stroke simplification 47 4.1 Abstract 48 4.2 Introduction 48 4.3 Detection of linear building groups 50 4.3.1 Stroke-based detection method 50 4.3.2 Distinguishing collinear and curvilinear patterns 53 4.4 Typification method 55 4.4.1 Analogy of building typification and line simplification 55 4.4.2 Stroke generation 56 4.4.3 Stroke simplification 57 4.5 Representation of newly typified buildings 60 4.6 Experiment 63 4.6.1 Linear building group detection 63 4.6.2 Typification results 65 4.7 Discussion 66 4.7.1 Comparison of reallocating remained nodes 66 4.7.2 Comparison with classic line simplification method 67 4.7.3 Advantage 69 4.7.4 Further improvement 71 4.8 Conclusion 71 Chapter 5 A mesh-based typification method for building groups with grid patterns 73 5.1 Abstract 74 5.2 Introduction 74 5.3 Related work 75 5.4 Methodology of mesh-based typification 78 5.4.1 Grid pattern classification 78 5.4.2 Mesh generation 79 5.4.3 Triangular mesh elimination 80 5.4.4 Number and positioning of typified buildings 82 5.4.5 Representation of typified buildings 83 5.4.6 Resizing Newly Typified Buildings 85 5.5 Experiments 86 5.5.1 Data derivation 86 5.5.2 Typification results and evaluation 87 5.5.3 Comparison with official map 91 5.6 Discussion 92 5.6.1 Advantages 92 5.6.2 Further improvements 93 5.7 Conclusion 94 Chapter 6 Hierarchical extraction of skeleton structures from discrete buildings 95 6.1 Abstract 96 6.2 Introduction 96 6.3 Related work 97 6.4 Study area 99 6.5 Hierarchical extraction of skeleton structures 100 6.5.1 Proximity Graph Network (PGN) of buildings 100 6.5.2 Centrality analysis of proximity graph network 103 6.5.3 Hierarchical skeleton structures of buildings 108 6.6 Generalization application 111 6.7 Experiment and discussion 114 6.7.1 Data statement 114 6.7.2 Experimental results 115 6.7.3 Discussion 118 6.8 Conclusions 120 Chapter 7 Discussion 121 7.1 Revisiting the research problems 121 7.2 Evaluation of the presented methodology 123 7.2.1 Strengths 123 7.2.2 Limitations 125 Chapter 8 Conclusions 127 8.1 Main contributions 127 8.2 Outlook 128 8.3 Final thoughts 131 Bibliography 132 Acknowledgements 142 Publications 14

Computer vision based techniques for fall detection with application towards assisted living

In this thesis, new computer vision based techniques are proposed to detect falls of an elderly person living alone. This is an important problem in assisted living. Different types of information extracted from video recordings are exploited for fall detection using both analytical and machine learning techniques. Initially, a particle filter is used to extract a 2D cue, head velocity, to determine a likely fall event. The human body region is then extracted with a modern background subtraction algorithm. Ellipse fitting is used to represent this shape and its orientation angle is employed for fall detection. An analytical method is used by setting proper thresholds against which the head velocity and orientation angle are compared for fall discrimination. Movement amplitude is then integrated into the fall detector to reduce false alarms. Since 2D features can generate false alarms and are not invariant to different directions, more robust 3D features are next extracted from a 3D person representation formed from video measurements from multiple calibrated cameras. Instead of using thresholds, different data fitting methods are applied to construct models corresponding to fall activities. These are then used to distinguish falls and non-falls. In the final works, two practical fall detection schemes which use only one un-calibrated camera are tested in a real home environment. These approaches are based on 2D features which describe human body posture. These extracted features are then applied to construct either a supervised method for posture classification or an unsupervised method for abnormal posture detection. Certain rules which are set according to the characteristics of fall activities are lastly used to build robust fall detection methods. Extensive evaluation studies are included to confirm the efficiency of the schemes