A survey on real-time 3D scene reconstruction with SLAM methods in embedded systems

The 3D reconstruction of simultaneous localization and mapping (SLAM) is an important topic in the field for transport systems such as drones, service robots and mobile AR/VR devices. Compared to a point cloud representation, the 3D reconstruction based on meshes and voxels is particularly useful for high-level functions, like obstacle avoidance or interaction with the physical environment. This article reviews the implementation of a visual-based 3D scene reconstruction pipeline on resource-constrained hardware platforms. Real-time performances, memory management and low power consumption are critical for embedded systems. A conventional SLAM pipeline from sensors to 3D reconstruction is described, including the potential use of deep learning. The implementation of advanced functions with limited resources is detailed. Recent systems propose the embedded implementation of 3D reconstruction methods with different granularities. The trade-off between required accuracy and resource consumption for real-time localization and reconstruction is one of the open research questions identified and discussed in this paper

Variational methods and its applications to computer vision

Many computer vision applications such as image segmentation can be formulated in a ''variational'' way as energy minimization problems. Unfortunately, the computational task of minimizing these energies is usually difficult as it generally involves non convex functions in a space with thousands of dimensions and often the associated combinatorial problems are NP-hard to solve. Furthermore, they are ill-posed inverse problems and therefore are extremely sensitive to perturbations (e.g. noise). For this reason in order to compute a physically reliable approximation from given noisy data, it is necessary to incorporate into the mathematical model appropriate regularizations that require complex computations. The main aim of this work is to describe variational segmentation methods that are particularly effective for curvilinear structures. Due to their complex geometry, classical regularization techniques cannot be adopted because they lead to the loss of most of low contrasted details. In contrast, the proposed method not only better preserves curvilinear structures, but also reconnects some parts that may have been disconnected by noise. Moreover, it can be easily extensible to graphs and successfully applied to different types of data such as medical imagery (i.e. vessels, hearth coronaries etc), material samples (i.e. concrete) and satellite signals (i.e. streets, rivers etc.). In particular, we will show results and performances about an implementation targeting new generation of High Performance Computing (HPC) architectures where different types of coprocessors cooperate. The involved dataset consists of approximately 200 images of cracks, captured in three different tunnels by a robotic machine designed for the European ROBO-SPECT project.Open Acces

Some Optimally Adaptive Parallel Graph Algorithms on EREW PRAM Model

The study of graph algorithms is an important area of research in computer science, since graphs offer useful tools to model many real-world situations. The commercial availability of parallel computers have led to the development of efficient parallel graph algorithms. Using an exclusive-read and exclusive-write (EREW) parallel random access machine (PRAM) as the computation model with a fixed number of processors, we design and analyze parallel algorithms for seven undirected graph problems, such as, connected components, spanning forest, fundamental cycle set, bridges, bipartiteness, assignment problems, and approximate vertex coloring. For all but the last two problems, the input data structure is an unordered list of edges, and divide-and-conquer is the paradigm for designing algorithms. One of the algorithms to solve the assignment problem makes use of an appropriate variant of dynamic programming strategy. An elegant data structure, called the adjacency list matrix, used in a vertex-coloring algorithm avoids the sequential nature of linked adjacency lists. Each of the proposed algorithms achieves optimal speedup, choosing an optimal granularity (thus exploiting maximum parallelism) which depends on the density or the number of vertices of the given graph. The processor-(time)2 product has been identified as a useful parameter to measure the cost-effectiveness of a parallel algorithm. We derive a lower bound on this measure for each of our algorithms

Design and test of a Displacement Workspace Mapping Station for articular joints

In 2003, 267,000 Americans received total knee replacements prohibiting high impact athletics for the remainder of a patient’s life. A better understanding of the movement and constraint of the knee is necessary to provide more realistic motion of or possibly eliminate the need for joint prosthetics. Fixed Orientation Displacement Workspaces (FODW) can be applied to study the relationship of the passive constraint system and six (6) degree of freedom (DOF) movement of the human knee. A FODW consists of the volume of possible positions the tibia/fibula can occupy relative to a fixed femur without changing the relative orientation of the bones. Theoretical models of the FODW provided a promising snapshot of knee kinematics. A Displacement Workspace Test Station (DWTS) for mapping FODWs was built. An in vitro articular joint completes the loop between a strain gauge-based six (6) axis load cell and a 6 DOF manipulandum mounted to a fixed reference frame. The joint is hand manipulated while a C++ program, Armtalk, operates applications that sample and filter both manipulandum position/orientation and load cell output signals at over 500Hz. Armtalk automatically stores raw data points at 2 Hz or upon a user foot-pedal signal. Forces and moments acting at the joint and its angular orientation are added to each raw data point by algorithms in a spreadsheet. The algorithms select points that represent a particular FODW according to a user specified range of acceptable joint forces and moments and bone orientations. The Cartesian coordinates of individual FODW data points are input into a NURBS-based CAD program for visualization. The DWTS has a 0.2286 mm positional accuracy, a 200 N capacity, and a 0.075 mm/kN compliance. A 2 DOF test checked the Armtalk application and calculated the DWTS angular accuracy to be 0.008°. To calibrate the load cell, moment and force scaling factors of 0.00922 in lb/unit and 0.00554 lb/unit were calculated, respectively. The spreadsheet algorithms successfully reduced data in a 6 DOF test. The CAD program modeled workspaces from 2 and 6 DOF tests with a 1.3 % volumetric accuracy. The apparatus is ready to map FODW of articular joints

Machine Learning for Instance Segmentation

Volumetric Electron Microscopy images can be used for connectomics, the study of brain connectivity at the cellular level. A prerequisite for this inquiry is the automatic identification of neural cells, which requires machine learning algorithms and in particular efficient image segmentation algorithms. In this thesis, we develop new algorithms for this task. In the first part we provide, for the first time in this field, a method for training a neural network to predict optimal input data for a watershed algorithm. We demonstrate its superior performance compared to other segmentation methods of its category. In the second part, we develop an efficient watershed-based algorithm for weighted graph partitioning, the \emph{Mutex Watershed}, which uses negative edge-weights for the first time. We show that it is intimately related to the multicut and has a cutting edge performance on a connectomics challenge. Our algorithm is currently used by the leaders of two connectomics challenges. Finally, motivated by inpainting neural networks, we create a method to learn the graph weights without any supervision

Efficient Algorithms for a Mesh-Connected Computer with Additional Global Bandwidth

This thesis shows that adding additional global bandwidths to a mesh-connected computer can greatly improve the performance. The goal of this project is to design algorithms for mesh-connected computers augmented with limited global bandwidth, so that we can further enhance our understanding of the parallel/serial nature of the problems on evolving parallel architectures. We do this by first solving several problems associated with fundamental data movement, then summarize ways to resolve different situations one may observe in data movement in parallel computing. This can help us to understand whether the problem is easily parallelizable on different parallel models. We give efficient algorithms to solve several fundamental problems, which include sorting, counting, fast Fourier transform, finding a minimum spanning tree, finding a convex hull, etc. We show that adding a small amount of global bandwidth makes a practical design that combines aspects of mesh and fully connected models to achieve the benefits of each. Most of the algorithms are optimal. For future work, we believe that algorithms with peak-power constrains can make our model well adapted to the recent architectures in high performance computing.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/150001/1/anyujie_1.pd

Micro/Nano Manufacturing

Micro manufacturing involves dealing with the fabrication of structures in the size range of 0.1 to 1000 µm. The scope of nano manufacturing extends the size range of manufactured features to even smaller length scales—below 100 nm. A strict borderline between micro and nano manufacturing can hardly be drawn, such that both domains are treated as complementary and mutually beneficial within a closely interconnected scientific community. Both micro and nano manufacturing can be considered as important enablers for high-end products. This Special Issue of Applied Sciences is dedicated to recent advances in research and development within the field of micro and nano manufacturing. The included papers report recent findings and advances in manufacturing technologies for producing products with micro and nano scale features and structures as well as applications underpinned by the advances in these technologies

Integrasjon av et minimalistisk sett av sensorer for kartlegging og lokalisering av landbruksroboter

Robots have recently become ubiquitous in many aspects of daily life. For in-house applications there is vacuuming, mopping and lawn-mowing robots. Swarms of robots have been used in Amazon warehouses for several years. Autonomous driving cars, despite being set back by several safety issues, are undeniably becoming the standard of the automobile industry. Not just being useful for commercial applications, robots can perform various tasks, such as inspecting hazardous sites, taking part in search-and-rescue missions. Regardless of end-user applications, autonomy plays a crucial role in modern robots. The essential capabilities required for autonomous operations are mapping, localization and navigation. The goal of this thesis is to develop a new approach to solve the problems of mapping, localization, and navigation for autonomous robots in agriculture. This type of environment poses some unique challenges such as repetitive patterns, large-scale sparse features environments, in comparison to other scenarios such as urban/cities, where the abundance of good features such as pavements, buildings, road lanes, traffic signs, etc., exists. In outdoor agricultural environments, a robot can rely on a Global Navigation Satellite System (GNSS) to determine its whereabouts. It is often limited to the robot's activities to accessible GNSS signal areas. It would fail for indoor environments. In this case, different types of exteroceptive sensors such as (RGB, Depth, Thermal) cameras, laser scanner, Light Detection and Ranging (LiDAR) and proprioceptive sensors such as Inertial Measurement Unit (IMU), wheel-encoders can be fused to better estimate the robot's states. Generic approaches of combining several different sensors often yield superior estimation results but they are not always optimal in terms of cost-effectiveness, high modularity, reusability, and interchangeability. For agricultural robots, it is equally important for being robust for long term operations as well as being cost-effective for mass production. We tackle this challenge by exploring and selectively using a handful of sensors such as RGB-D cameras, LiDAR and IMU for representative agricultural environments. The sensor fusion algorithms provide high precision and robustness for mapping and localization while at the same time assuring cost-effectiveness by employing only the necessary sensors for a task at hand. In this thesis, we extend the LiDAR mapping and localization methods for normal urban/city scenarios to cope with the agricultural environments where the presence of slopes, vegetation, trees render the traditional approaches to fail. Our mapping method substantially reduces the memory footprint for map storing, which is important for large-scale farms. We show how to handle the localization problem in dynamic growing strawberry polytunnels by using only a stereo visual-inertial (VI) and depth sensor to extract and track only invariant features. This eliminates the need for remapping to deal with dynamic scenes. Also, for a demonstration of the minimalistic requirement for autonomous agricultural robots, we show the ability to autonomously traverse between rows in a difficult environment of zigzag-liked polytunnel using only a laser scanner. Furthermore, we present an autonomous navigation capability by using only a camera without explicitly performing mapping or localization. Finally, our mapping and localization methods are generic and platform-agnostic, which can be applied to different types of agricultural robots. All contributions presented in this thesis have been tested and validated on real robots in real agricultural environments. All approaches have been published or submitted in peer-reviewed conference papers and journal articles.Roboter har nylig blitt standard i mange deler av hverdagen. I hjemmet har vi støvsuger-, vaske- og gressklippende roboter. Svermer med roboter har blitt brukt av Amazons varehus i mange år. Autonome selvkjørende biler, til tross for å ha vært satt tilbake av sikkerhetshensyn, er udiskutabelt på vei til å bli standarden innen bilbransjen. Roboter har mer nytte enn rent kommersielt bruk. Roboter kan utføre forskjellige oppgaver, som å inspisere farlige områder og delta i leteoppdrag. Uansett hva sluttbrukeren velger å gjøre, spiller autonomi en viktig rolle i moderne roboter. De essensielle egenskapene for autonome operasjoner i landbruket er kartlegging, lokalisering og navigering. Denne type miljø gir spesielle utfordringer som repetitive mønstre og storskala miljø med få landskapsdetaljer, sammenlignet med andre steder, som urbane-/bymiljø, hvor det finnes mange landskapsdetaljer som fortau, bygninger, trafikkfelt, trafikkskilt, etc. I utendørs jordbruksmiljø kan en robot bruke Global Navigation Satellite System (GNSS) til å navigere sine omgivelser. Dette begrenser robotens aktiviteter til områder med tilgjengelig GNSS signaler. Dette vil ikke fungere i miljøer innendørs. I ett slikt tilfelle vil reseptorer mot det eksterne miljø som (RGB-, dybde-, temperatur-) kameraer, laserskannere, «Light detection and Ranging» (LiDAR) og propriopsjonære detektorer som treghetssensorer (IMU) og hjulenkodere kunne brukes sammen for å bedre kunne estimere robotens tilstand. Generisk kombinering av forskjellige sensorer fører til overlegne estimeringsresultater, men er ofte suboptimale med hensyn på kostnadseffektivitet, moduleringingsgrad og utbyttbarhet. For landbruksroboter så er det like viktig med robusthet for lang tids bruk som kostnadseffektivitet for masseproduksjon. Vi taklet denne utfordringen med å utforske og selektivt velge en håndfull sensorer som RGB-D kameraer, LiDAR og IMU for representative landbruksmiljø. Algoritmen som kombinerer sensorsignalene gir en høy presisjonsgrad og robusthet for kartlegging og lokalisering, og gir samtidig kostnadseffektivitet med å bare bruke de nødvendige sensorene for oppgaven som skal utføres. I denne avhandlingen utvider vi en LiDAR kartlegging og lokaliseringsmetode normalt brukt i urbane/bymiljø til å takle landbruksmiljø, hvor hellinger, vegetasjon og trær gjør at tradisjonelle metoder mislykkes. Vår metode reduserer signifikant lagringsbehovet for kartlagring, noe som er viktig for storskala gårder. Vi viser hvordan lokaliseringsproblemet i dynamisk voksende jordbær-polytuneller kan løses ved å bruke en stereo visuel inertiel (VI) og en dybdesensor for å ekstrahere statiske objekter. Dette eliminerer behovet å kartlegge på nytt for å klare dynamiske scener. I tillegg demonstrerer vi de minimalistiske kravene for autonome jordbruksroboter. Vi viser robotens evne til å bevege seg autonomt mellom rader i ett vanskelig miljø med polytuneller i sikksakk-mønstre ved bruk av kun en laserskanner. Videre presenterer vi en autonom navigeringsevne ved bruk av kun ett kamera uten å eksplisitt kartlegge eller lokalisere. Til slutt viser vi at kartleggings- og lokaliseringsmetodene er generiske og platform-agnostiske, noe som kan brukes med flere typer jordbruksroboter. Alle bidrag presentert i denne avhandlingen har blitt testet og validert med ekte roboter i ekte landbruksmiljø. Alle forsøk har blitt publisert eller sendt til fagfellevurderte konferansepapirer og journalartikler

Energy-Efficient Algorithms on Mesh-Connected Systems with Additional Communication Links.

Energy consumption has become a critical factor constraining the design of massively parallel computers, necessitating the development of new models and energy-efficient algorithms. In this work we take a fundamental abstract model of massive parallelism, the mesh-connected computer, and extend it with additional communication links motivated by recent advances in on-chip photonic interconnects. This new means of communication with optical signals rather than electrical signals can reduce the energy and/or time of calculations by providing faster communication between distant processing elements. Processors are arranged in a two-dimensional grid with wire connections between adjacent neighbors and an additional one or two layers of noncrossing optical connections. Varying constraints on the layout of optics affect how powerful the model can be. In this dissertation, three optical interconnection layouts are defined: the optical mesh, the optical mesh of trees, and the optical pyramid. For each layout, algorithms for solving important problems are presented. Since energy usage is an important factor, running times are given in terms of a peak-power constraint, where peak power is the maximum number of processors active at any one time. These results demonstrate advantages of optics in terms of improved time and energy usage over the standard mesh computer without optics. One of the most significant results shows an optimal nonlinear time/peak-power tradeoff for sorting on the optical pyramid. This work shows asymptotic theoretical limits of computation and energy usage on an abstract model which takes physical constraints and developing interconnection technology into account.PhDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/102474/1/ppoon_1.pd

An evaluation of the mechanical behaviour of imperfect aluminium tubes

Energy absorption mechanisms have been investigated intensively for the past decades by various authors and institutions, and numerous articles and other literature sources are available in print, as well as on the Internet. Energy absorbers and crashworthiness structures are two main research components in the energy absorption field under investigation today. In this research geometric changes are introduced on Al 6063-T6 circular tubes in the form of horizontal and spiral grooves to asses their influence on energy absorption characteristics. The horizontal and spiral grooves were cut into the tube to a cut depth of half the wall thickness of the tubes. The pitch was varied for both the horizontal and spiral grooves, while the cut width was kept constant. A specially designed static impact sleeve was used to compress the test specimens axially in an Instron 250 kN universal hydraulic testing system. Load vs. displacement graphs were generated from the captured experimental data for the uncut, horizontal and spiral grooved tubes. Energy vs. displacement graphs were created from the experimental data. The final deformed tubes were visually examined to determine the effect the geometric change had on the circular tube form, as well as the deformation pattern of the crushed tube. A Finite Element Method model is presented for each of the experimentally investigated tube impact models. A two dimensional (2D) model for the uncut as well as horizontally grooved tube is generated and analysed using a quasi static loading approach. Non-linear material properties are assigned to the model, and the Riks algorithm is used to model the non-linear post buckling behaviour of the various tubes. The results from the FEM analysis are used to generate load vs. displacement and energy vs. displacement graphs that are compared with the experimental data. Three dimensional (3D) FEM models of the normal, spiral and horizontal cut tubes were also generated in a CAD environment. A dynamic explicit non-linear analysis was done for each of the models to determine the reaction force and energy output values of each of the models. All analyses extend into the plastic material domain. Reaction force vs. displacement and energy vs. displacement graphs are generated from these analyses. A comparison is made between the numerically and experimentally determined gradients of the energy vs. displacement graphs of each of the tubes investigated. This forms the basis for an energy absorber design with application in the transport industry. Unique geometric imperfections were investigated experimentally and numerically for aluminium tubes. A lower buckling load than that for the normal tubes was achieved with the introduction of these geometric imperfections. New deformation patterns on tubes with imperfections not previously observed were described and analysed extensively. The load vs displacement graphs showed a constant increase in the load for the spiral grooved tubes. From the comparison between the numerically and experimentally investigated geometric imperfections a design guide line was esthablished and used in the conceptual design of an energy absorber for the automotive industry.Prof. L. Pretorius Prof. R.F. Laubsche