Large-Scale Textured 3D Scene Reconstruction

Die Erstellung dreidimensionaler Umgebungsmodelle ist eine fundamentale Aufgabe im Bereich des maschinellen Sehens. Rekonstruktionen sind für eine Reihe von Anwendungen von Nutzen, wie bei der Vermessung, dem Erhalt von Kulturgütern oder der Erstellung virtueller Welten in der Unterhaltungsindustrie. Im Bereich des automatischen Fahrens helfen sie bei der Bewältigung einer Vielzahl an Herausforderungen. Dazu gehören Lokalisierung, das Annotieren großer Datensätze oder die vollautomatische Erstellung von Simulationsszenarien. Die Herausforderung bei der 3D Rekonstruktion ist die gemeinsame Schätzung von Sensorposen und einem Umgebunsmodell. Redundante und potenziell fehlerbehaftete Messungen verschiedener Sensoren müssen in eine gemeinsame Repräsentation der Welt integriert werden, um ein metrisch und photometrisch korrektes Modell zu erhalten. Gleichzeitig muss die Methode effizient Ressourcen nutzen, um Laufzeiten zu erreichen, welche die praktische Nutzung ermöglichen. In dieser Arbeit stellen wir ein Verfahren zur Rekonstruktion vor, das fähig ist, photorealistische 3D Rekonstruktionen großer Areale zu erstellen, die sich über mehrere Kilometer erstrecken. Entfernungsmessungen aus Laserscannern und Stereokamerasystemen werden zusammen mit Hilfe eines volumetrischen Rekonstruktionsverfahrens fusioniert. Ringschlüsse werden erkannt und als zusätzliche Bedingungen eingebracht, um eine global konsistente Karte zu erhalten. Das resultierende Gitternetz wird aus Kamerabildern texturiert, wobei die einzelnen Beobachtungen mit ihrer Güte gewichtet werden. Für eine nahtlose Erscheinung werden die unbekannten Belichtungszeiten und Parameter des optischen Systems mitgeschätzt und die Bilder entsprechend korrigiert. Wir evaluieren unsere Methode auf synthetischen Daten, realen Sensordaten unseres Versuchsfahrzeugs und öffentlich verfügbaren Datensätzen. Wir zeigen qualitative Ergebnisse großer innerstädtischer Bereiche, sowie quantitative Auswertungen der Fahrzeugtrajektorie und der Rekonstruktionsqualität. Zuletzt präsentieren wir mehrere Anwendungen und zeigen somit den Nutzen unserer Methode für Anwendungen im Bereich des automatischen Fahrens

Deep Learning based 3D Segmentation: A Survey

3D object segmentation is a fundamental and challenging problem in computer vision with applications in autonomous driving, robotics, augmented reality and medical image analysis. It has received significant attention from the computer vision, graphics and machine learning communities. Traditionally, 3D segmentation was performed with hand-crafted features and engineered methods which failed to achieve acceptable accuracy and could not generalize to large-scale data. Driven by their great success in 2D computer vision, deep learning techniques have recently become the tool of choice for 3D segmentation tasks as well. This has led to an influx of a large number of methods in the literature that have been evaluated on different benchmark datasets. This paper provides a comprehensive survey of recent progress in deep learning based 3D segmentation covering over 150 papers. It summarizes the most commonly used pipelines, discusses their highlights and shortcomings, and analyzes the competitive results of these segmentation methods. Based on the analysis, it also provides promising research directions for the future.Comment: Under review of ACM Computing Surveys, 36 pages, 10 tables, 9 figure

IoTにおけるリソースの最適化

Recently, there are more than 9 billion things that connected in the Internet of Things (IoT) and the number is exceed more than 24 billion in 2020. It means that numerous data will be generated because of increasing quickly of the number of things. An infrastructure should be developed to manage the connected things in IoT. Moreover, cloud computing will play important role in terms of data storage and analysis for IoT. Therefore, a cloud broker is considered as an intermediary in the infrastructure for managing the connected things. The cloud broker will find the best deal between clients and service providers. However, there are three problems among cloud broker, clients and service providers that are the response time of the request from clients, the energy consumption of the system and the profit of the cloud broker. The three problems are considered as multi-objective optimization problem to maximize the profit of the broker while minimizing the response time of the request and the energy consumption. A multi-objective particle swarm optimization (MOPSO) is proposed to solve the problem. MOPSO is compared with a non-dominated sorting genetic algorithm-II (NSGA-II) and a random search algorithm to show the performance. Since, there are a lot of data including social media and geographic location, generated in IoT. Coupling social media with geographic location has boosted the worth of understanding the real-world situations. Event detection aims to find more specific topic which represents real-world event. However,identification of unusual and seemingly inconsistent patterns in data, called outliers, is necessary. The problem is how to partition a spatio-temporal domain to find a meaningful local outlier pattern. A k-dimensional (KD) tree partitioning is applied to divide a spatio-temporal domain into sub-cells. The optimal partitioning problem in a spatio-temporal domain has been proven as an NP-complete problem. Therefore, a genetic algorithm is proposed to solve the problem. Moreover, the smart grid is strongly related to IoT technologies. It is enabled by IoT to handle big data and reduce the number of communication protocols. The micro-grid is studied because of micro-grids are part of a larger system that makes the smart grid to become reality. The operation management problem and pollutant emission problem are important problems for the micro-grid system. Thus, reducing the total energy expenses and pollutant emission of micro-grid and improving the renewable energy sources (battery energy storage) are considered together with the operation management of the micro-grid system. A fitness-based modified game particle swarm optimization (FMGPSO) algorithm is proposed to minimize the total costs of operation and pollutant emissions in the microgrid and multi-microgrid system. FMGPSO is compared with A non-dominated sorting genetic algorithm-III (NSGA-III), a multi-objective covariance matrix adaptation evolution strategy (MO-CMAES), and a speed-constrained multi-objective particle swarm optimization (SMPSO) to show the performance.最近では、Internet of Things（IoT）に接続されているものは90億件を超え、2020年には240億件を超えている。それは、物事の数が急速に増えるため、多くのデータが生成されることを意味する。IoTで接続されたものを管理するためのインフラストラクチャを開発する必要がある。さらに、クラウドコンピューティングは、IoTのデータストレージと分析の観点から重要な役割を果たしている。したがって、クラウドブローカーは、接続されたものを管理するためのインフラストラクチャの仲介者とみなされる。クラウドブローカーは、クライアントとサービスプロバイダーの間で最良の取引を見つけるだろう。しかし、クラウドブローカー、クライアントおよびサービスプロバイダーには、クライアントからの要求の応答時間、システムのエネルギー消費、クラウドブローカーのプロセスという3つの問題がある。この3つの問題は、要求の応答時間とエネルギー消費を最小限に抑えながら、ブローカーのプロビジョニングを最大化するための多目的最適化問題とみなされる。この問題を解決するために、多目的粒子群最適化（MOPSO）が提案されている。 MOPSOは、非優性選別遺伝的アルゴリズム-II（NSGA-II）およびランダム探索アルゴリズムと比較され、性能が示される。ソーシャルメディアや地理的な場所など、IoTで生成される多くのデータがあるためである。地理的な場所とソーシャルメディアを結び付けることで、現実の状況を理解する価値が高まっている。イベント検出は、実際のイベントを表すより特定のトピックを見つけることを目指している。しかし、異常値と呼ばれる異常なパターンや一見不整合なパターンの同定が必要である。問題は、時空間ドメインを分割して意味のある局所的な奇妙なパターンを見つける方法である。時空間領域をサブセルに分割するために、k次元（KD）ツリー分割が適用される。時空間領域における最適な分割問題は、NP完全な問題として証明されている。したがって、この問題を解決するための遺伝的アルゴリズムが提案されている。さらに、スマートグリッドはIoT技術と強く関連している。 IoTによって大きなデータを処理し、通信プロトコルの数を減らすことができる。マイクログリッドはスマートグリッドを現実化させるより大きなシステムの一部であるため、マイクログリッドが研究されている。運用管理上の問題や汚染物質排出問題は、マイクログリッドシステムにとって重要な問題である。したがって、マイクログリッドシステムの運用管理とともに、マイクログリッドの総エネルギー費用と汚染物質排出量の削減と再生可能エネルギー源の改善（バッテリエネルギー貯蔵）が考慮されている。マイクログリッドおよびマルチマイクログリッドシステムにおける操作および汚染物質排出の総コストを最小限に抑えるため、MGPSO アルゴリズムが提案されている。 FMGPSOは、非優先ソート遺伝的アルゴリズム-III（NSGA-III）、多目的共分散行列適応進化戦略（MO-CMAES）、および性能を示すために速度が制約された多目的粒子群最適化（SMPSO）と比較される。室蘭工業大学 (Muroran Institute of Technology)博士（工学

Feature-based hybrid inspection planning for complex mechanical parts

Globalization and emerging new powers in the manufacturing world are among many challenges, major manufacturing enterprises are facing. This resulted in increased alternatives to satisfy customers\u27 growing needs regarding products\u27 aesthetic and functional requirements. Complexity of part design and engineering specifications to satisfy such needs often require a better use of advanced and more accurate tools to achieve good quality. Inspection is a crucial manufacturing function that should be further improved to cope with such challenges. Intelligent planning for inspection of parts with complex geometric shapes and free form surfaces using contact or non-contact devices is still a major challenge. Research in segmentation and localization techniques should also enable inspection systems to utilize modern measurement technologies capable of collecting huge number of measured points. Advanced digitization tools can be classified as contact or non-contact sensors. The purpose of this thesis is to develop a hybrid inspection planning system that benefits from the advantages of both techniques. Moreover, the minimization of deviation of measured part from the original CAD model is not the only characteristic that should be considered when implementing the localization process in order to accept or reject the part; geometric tolerances must also be considered. A segmentation technique that deals directly with the individual points is a necessary step in the developed inspection system, where the output is the actual measured points, not a tessellated model as commonly implemented by current segmentation tools. The contribution of this work is three folds. First, a knowledge-based system was developed for selecting the most suitable sensor using an inspection-specific features taxonomy in form of a 3D Matrix where each cell includes the corresponding knowledge rules and generate inspection tasks. A Travel Salesperson Problem (TSP) has been applied for sequencing these hybrid inspection tasks. A novel region-based segmentation algorithm was developed which deals directly with the measured point cloud and generates sub-point clouds, each of which represents a feature to be inspected and includes the original measured points. Finally, a new tolerance-based localization algorithm was developed to verify the functional requirements and was applied and tested using form tolerance specifications. This research enhances the existing inspection planning systems for complex mechanical parts with a hybrid inspection planning model. The main benefits of the developed segmentation and tolerance-based localization algorithms are the improvement of inspection decisions in order not to reject good parts that would have otherwise been rejected due to misleading results from currently available localization techniques. The better and more accurate inspection decisions achieved will lead to less scrap, which, in turn, will reduce the product cost and improve the company potential in the market

A practical comparison between two powerful PCC codec’s

Recent advances in the consumption of 3D content creates the necessity of efficient ways to visualize and transmit 3D content. As a result, methods to obtain that same content have been evolving, leading to the development of new methods of representations, namely point clouds and light fields. A point cloud represents a set of points with associated Cartesian coordinates associated with each point(x, y, z), as well as being able to contain even more information inside that point (color, material, texture, etc). This kind of representation changes the way on how 3D content in consumed, having a wide range of applications, from videogaming to medical ones. However, since this type of data carries so much information within itself, they are data-heavy, making the storage and transmission of content a daunting task. To resolve this issue, MPEG created a point cloud coding normalization project, giving birth to V-PCC (Video-based Point Cloud Coding) and G-PCC (Geometry-based Point Cloud Coding) for static content. Firstly, a general analysis of point clouds is made, spanning from their possible solutions, to their acquisition. Secondly, point cloud codecs are studied, namely VPCC and G-PCC from MPEG. Then, a state of art study of quality evaluation is performed, namely subjective and objective evaluation. Finally, a report on the JPEG Pleno Point Cloud, in which an active colaboration took place, is made, with the comparative results of the two codecs and used metrics.Os avanços recentes no consumo de conteúdo 3D vêm criar a necessidade de maneiras eficientes de visualizar e transmitir conteúdo 3D. Consequentemente, os métodos de obtenção desse mesmo conteúdo têm vindo a evoluir, levando ao desenvolvimento de novas maneiras de representação, nomeadamente point clouds e lightfields. Um point cloud (núvem de pontos) representa um conjunto de pontos com coordenadas cartesianas associadas a cada ponto (x, y, z), além de poder conter mais informação dentro do mesmo (cor, material, textura, etc). Este tipo de representação abre uma nova janela na maneira como se consome conteúdo 3D, tendo um elevado leque de aplicações, desde videojogos e realidade virtual a aplicações médicas. No entanto, este tipo de dados, ao carregarem com eles tanta informação, tornam-se incrivelmente pesados, tornando o seu armazenamento e transmissão uma tarefa hercúleana. Tendo isto em mente, a MPEG criou um projecto de normalização de codificação de point clouds, dando origem ao V-PCC (Video-based Point Cloud Coding) e G-PCC (Geometry-based Point Cloud Coding) para conteúdo estático. Esta dissertação tem como objectivo uma análise geral sobre os point clouds, indo desde as suas possívei utilizações à sua aquisição. Seguidamente, é efectuado um estudo dos codificadores de point clouds, nomeadamente o V-PCC e o G-PCC da MPEG, o estado da arte da avaliação de qualidade, objectiva e subjectiva, e finalmente, são reportadas as actividades da JPEG Pleno Point Cloud, na qual se teve uma colaboração activa

Probabilistic best-fit multi-dimensional range query in Self-Organizing Cloud

With virtual machine (VM) technology being increasingly mature, computing resources in modern Cloud systems can be partitioned in fine granularity and allocated on demand with 'pay-as-you-go' model. In this work, we study the resource query and allocation problems in a Self- Organizing Cloud (SOC), where host machines are connected by a peer-to-peer (P2P) overlay network on the Internet. To run a user task in SOC, the requester needs to perform a multi-dimensional range search over the P2P network for locating host machines that satisfy its minimal demand on each type of resources. The multi-dimensional range search problem is known to be challenging as contentions along multiple dimensions could happen in the presence of the uncoordinated analogous queries. Moreover, low resource matching rate may happen while restricting query delay and network traffic. We design a novel resource discovery protocol, namely Proactive Index Diffusion CAN (PID-CAN), which can proactively diffuse resource indexes over the nodes and randomly route query messages among them. Such a protocol is especially suitable for the range query that needs to maximize its best-fit resource shares under possible competition along multiple resource dimensions. Via simulation, we show that PID-CAN could keep stable and optimized searching performance with low query delay and traffic overhead, for various test cases under different distributions of query ranges and competition degrees. It also performs satisfactorily in dynamic node-churning situation. © 2011 IEEE.published_or_final_versionThe 40th International Conference on Parallel Processing (ICPP-2011), Taipei City, Taiwan, 13-16 September 2011. In Proceedings of the 40th ICPP, 2011, p. 763-77

Optimal and probabilistic resource and capability analysis for network slice as a service

Network Slice as a Service is one of the key concepts of the fifth generation of mobile networks (5G). 5G supports new use cases, like the Internet of Things (IoT), massive Machine Type Communication (mMTC) and Ultra-Reliable and Low Latency Communication (URLLC) as well as significant improvements of the conventional Mobile Broadband (MBB) use case. In addition, safety and security critical use cases move into focus. These use cases involve diverging requirements, e.g. network reliability, latency and throughput. Network virtualization and end-to-end mobile network slicing are seen as key enablers to handle those differing requirements and providing mobile network services for the various 5G use cases and between different tenants. Network slices are isolated, virtualized, end-to-end networks optimized for specific use cases. But still they share a common physical network infrastructure. Through logical separation of the network slices on a common end-to-end mobile network infrastructure, an efficient usage of the underlying physical network infrastructure provided by multiple Mobile Service Providers (MSPs) in enabled. Due to the dynamic lifecycle of network slices there is a strong demand for efficient algorithms for the so-called Network Slice Embedding (NSE) problem. Efficient and reliable resource provisioning for Network Slicing as a Service, requires resource allocation based on a mapping of virtual network slice elements on the serving physical mobile network infrastructure. In this thesis, first of all, a formal Network Slice Instance Admission (NSIA) process is presented, based on the 3GPP standardization. This process allows to give fast feedback to a network operator or tenant on the feasibility of embedding incoming Network Slice Instance Requests (NSI-Rs). In addition, corresponding services for NSIA and feasibility checking services are defined in the context of the ETSI ZSM Reference Architecture Framework. In the main part of this work, a mathematical model for solving the NSE Problem formalized as a standardized Linear Program (LP) is presented. The presented solution provides a nearly optimal embedding. This includes the optimal subset of Network Slice Instances (NSIs) to be selected for embedding, in terms of network slice revenue and costs, and the optimal allocation of associated network slice applications, functions, services and communication links on the 5G end-to-end mobile network infrastructure. It can be used to solve the online as well as the offline NSIA problem automatically in different variants. In particular, low latency network slices require deployment of their services and applications, including Network Functions (NFs) close to the user, i.e., at the edge of the mobile network. Since the users of those services might be widely distributed and mobile, multiple instances of the same application are required to be available on numerous distributed edge clouds. A holistic approach for tackling the problem of NSE with edge computing is provided by our so-called Multiple Application Instantiation (MAI) variant of the NSE LP solution. It is capable of determining the optimal number of application instances and their optimal deployment locations on the edge clouds, even for multiple User Equipment (UE) connectivity scenarios. In addition to that multi-path, also referred to as path-splitting, scenarios with a latency sensitive objective function, which guarantees the optimal network utilization as well as minimum latency in the network slice communication, is included. Resource uncertainty, as well as reuse and overbooking of resources guaranteed by Service Level Agreements (SLAs) are discussed in this work. There is a consensus that over-provisioning of mobile communication bands is economically infeasible and certain risk of network overload is accepted for the majority of the 5G use cases. A probabilistic variant of the NSE problem with an uncertainty-aware objective function and a resource availability confidence analysis are presented. The evaluation shows the advantages and the suitability of the different variants of the NSE formalization, as well as its scalability and computational limits in a practical implementation

Fine-scale Inventory of Forest Biomass with Ground-based LiDAR

Biomass measurement provides a baseline for ecosystem valuation required by modern forest management. The advent of ground-based LiDAR technology, renowned for 3D sampling resolution, has been altering the routines of biomass inventory. The thesis develops a set of innovative approaches in support of fine-scale biomass inventory, including automatic extraction of stem statistics, robust delineation of plot biomass components, accurate classification of individual tree species, and repeatable scanning of plot trees using a lightweight scanning system. Main achievements in terms of accuracy are a relative root mean square error of 11% for stem volume extraction, a mean classification accuracy of 0.72 for plot wood components, and a classification accuracy of 92% among seven tree species. The results indicate the technical feasibility of biomass delineation and monitoring from plot-level and multi-species point cloud datasets, whereas point occlusion and lack of fine-scale validation dataset are current challenges for biomass 3D analysis from ground.S.G.S. International Tuition Award from the University of Lethbridge The Dean's Scholarship from the University of Lethbridge Campus Alberta Innovates Program NSERC Discovery Grants Progra