As-Built 3D Heritage City Modelling to Support Numerical Structural Analysis: Application to the Assessment of an Archaeological Remain

Terrestrial laser scanning is a widely used technology to digitise archaeological, architectural and cultural heritage. This allows for modelling the assets’ real condition in comparison with traditional data acquisition methods. This paper, based on the case study of the basilica in the Baelo Claudia archaeological ensemble (Tarifa, Spain), justifies the need of accurate heritage modelling against excessively simplified approaches in order to support structural safety analysis. To do this, after validating the 3Dmeshing process frompoint cloud data, the semi-automatic digital reconstitution of the basilica columns is performed. Next, a geometric analysis is conducted to calculate the structural alterations of the columns. In order to determine the structural performance, focusing both on the accuracy and suitability of the geometric models, static and modal analyses are carried out by means of the finite element method (FEM) on three different models for the most unfavourable column in terms of structural damage: (1) as-built (2) simplified and (3) ideal model without deformations. Finally, the outcomes show that the as-built modelling enhances the conservation status analysis of the 3D heritage city (in terms of realistic compliance factor values), although further automation still needs to be implemented in the modelling process

Photogrammetric techniques for across-scale soil erosion assessment: Developing methods to integrate multi-temporal high resolution topography data at field plots

Soil erosion is a complex geomorphological process with varying influences of different impacts at different spatio-temporal scales. To date, measurement of soil erosion is predominantly realisable at specific scales, thereby detecting separate processes, e.g. interrill erosion contrary to rill erosion. It is difficult to survey soil surface changes at larger areal coverage such as field scale with high spatial resolution. Either net changes at the system outlet or remaining traces after the erosional event are usually measured. Thus, either quasi-point measurements are extrapolated to the corresponding area without knowing the actual sediment source as well as sediment storage behaviour on the plot or erosion rates are estimated disrupting the area of investigation during the data acquisition impeding multi-temporal assessment. Furthermore, established methods of soil erosion detection and quantification are typically only reliable for large event magnitudes, very labour and time intense, or inflexible. To better observe soil erosion processes at field scale and under natural conditions, the development of a method is necessary, which identifies and quantifies sediment sources and sinks at the hillslope with high spatial resolution and captures single precipitation events as well as allows for longer observation periods. Therefore, an approach is introduced, which measures soil surface changes for multi-spatio-temporal scales without disturbing the area of interest. Recent advances regarding techniques to capture high resolution topography (HiRT) data led to several promising tools for soil erosion measurement with corresponding advantages but also disadvantages. The necessity exists to evaluate those methods because they have been rarely utilised in soil surface studies. On the one hand, there is terrestrial laser scanning (TLS), which comprises high error reliability and retrieves 3D information directly. And on the other hand, there is unmanned aerial vehicle (UAV) technology in combination with structure from motion (SfM) algorithms resulting in UAV photogrammetry, which is very flexible in the field and depicts a beneficial perspective. Evaluation of the TLS feasibility reveals that this method implies a systematic error that is distance-related and temporal constant for the investigated device and can be corrected transferring calibration values retrieved from an estimated lookup table. However, TLS still reaches its application limits quickly due to an unfavourable (almost horizontal) scanning view at the soil surface resulting in a fast decrease of point density and increase of noise with increasing distance from the device. UAV photogrammetry allows for a better perspective (birds-eye view) onto the area of interest, but possesses more complex error behaviour, especially in regard to the systematic error of a DEM dome, which depends on the method for 3D reconstruction from 2D images (i.e. options for additional implementation of observations) and on the image network configuration (i.e. parallel-axes and control point configuration). Therefore, a procedure is developed that enables flexible usage of different cameras and software tools without the need of additional information or specific camera orientations and yet avoiding this dome error. Furthermore, the accuracy potential of UAV photogrammetry describing rough soil surfaces is assessed because so far corresponding data is missing. Both HiRT methods are used for multi-temporal measurement of soil erosion processes resulting in surface changes of low magnitudes, i.e. rill and especially interrill erosion. Thus, a reference with high accuracy and stability is a requirement. A local reference system with sub-cm and at its best 1 mm accuracy is setup and confirmed by control surveys. TLS and UAV photogrammetry data registration with these targets ensures that errors due to referencing are of minimal impact. Analysis of the multi-temporal performance of both HiRT methods affirms TLS to be suitable for the detection of erosion forms of larger magnitudes because of a level of detection (LoD) of 1.5 cm. UAV photogrammetry enables the quantification of even lower magnitude changes (LoD of 1 cm) and a reliable observation of the change of surface roughness, which is important for runoff processes, at field plots due to high spatial resolution (1 cm²). Synergetic data fusion as a subsequent post-processing step is necessary to exploit the advantages of both HiRT methods and potentially further increase the LoD. The unprecedented high level of information entails the need for automatic geomorphic feature extraction due to the large amount of novel content. Therefore, a method is developed, which allows for accurate rill extraction and rill parameter calculation with high resolution enabling new perspectives onto rill erosion that has not been possible before due to labour and area access limits. Erosion volume and cross sections are calculated for each rill revealing a dominant rill deepening. Furthermore, rill shifting in dependence of the rill orientation towards the dominant wind direction is revealed. Two field plots are installed at erosion prone positions in the Mediterranean (1,000 m²) and in the European loess belt (600 m²) to ensure the detection of surface changes, permitting the evaluation of the feasibility, potential and limits of TLS and UAV photogrammetry in soil erosion studies. Observations are made regarding sediment connectivity at the hillslope scale. Both HiRT methods enable the identification of local sediment sources and sinks, but still exhibiting some degree of uncertainty due to the comparable high LoD in regard to laminar accumulation and interrill erosion processes. At both field sites wheel tracks and erosion rills increase hydrological and sedimentological connectivity. However, at the Mediterranean field plot especially dis-connectivity is obvious. At the European loess belt case study a triggering event could be captured, which led to high erosion rates due to high soil moisture contents and yet further erosion increase due to rill amplification after rill incision. Estimated soil erosion rates range between 2.6 tha-1 and 121.5 tha-1 for single precipitation events and illustrate a large variability due to very different site specifications, although both case studies are located in fragile landscapes. However, the susceptibility to soil erosion has different primary causes, i.e. torrential precipitation at the Mediterranean site and high soil erodibility at the European loess belt site. The future capability of the HiRT methods is their potential to be applicable at yet larger scales. Hence, investigations of the importance of gullys for sediment connectivity between hillslopes and channels are possible as well as the possible explanation of different erosion rates observed at hillslope and at catchment scales because local sediment sink and sources can be quantified. In addition, HiRT data can be a great tool for calibrating, validating and enhancing soil erosion models due to the unprecedented level of detail and the flexible multi-spatio-temporal application

A 3D descriptor to detect task-oriented grasping points in clothing

© 2016. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/Manipulating textile objects with a robot is a challenging task, especially because the garment perception is difficult due to the endless configurations it can adopt, coupled with a large variety of colors and designs. Most current approaches follow a multiple re-grasp strategy, in which clothes are sequentially grasped from different points until one of them yields a recognizable configuration. In this work we propose a method that combines 3D and appearance information to directly select a suitable grasping point for the task at hand, which in our case consists of hanging a shirt or a polo shirt from a hook. Our method follows a coarse-to-fine approach in which, first, the collar of the garment is detected and, next, a grasping point on the lapel is chosen using a novel 3D descriptor. In contrast to current 3D descriptors, ours can run in real time, even when it needs to be densely computed over the input image. Our central idea is to take advantage of the structured nature of range images that most depth sensors provide and, by exploiting integral imaging, achieve speed-ups of two orders of magnitude with respect to competing approaches, while maintaining performance. This makes it especially adequate for robotic applications as we thoroughly demonstrate in the experimental section.Peer ReviewedPostprint (author's final draft

Time-of-flight-assisted Kinect camera-based people detection for intuitive human robot cooperation in the surgical operating room

Scene supervision is a major tool to make medical robots safer and more intuitive. The paper shows an approach to efficiently use 3D cameras within the surgical operating room to enable for safe human robot interaction and action perception. Additionally the presented approach aims to make 3D camera-based scene supervision more reliable and accurate

Geometric data understanding : deriving case specific features

There exists a tradition using precise geometric modeling, where uncertainties in data can be considered noise. Another tradition relies on statistical nature of vast quantity of data, where geometric regularity is intrinsic to data and statistical models usually grasp this level only indirectly. This work focuses on point cloud data of natural resources and the silhouette recognition from video input as two real world examples of problems having geometric content which is intangible at the raw data presentation. This content could be discovered and modeled to some degree by such machine learning (ML) approaches like deep learning, but either a direct coverage of geometry in samples or addition of special geometry invariant layer is necessary. Geometric content is central when there is a need for direct observations of spatial variables, or one needs to gain a mapping to a geometrically consistent data representation, where e.g. outliers or noise can be easily discerned. In this thesis we consider transformation of original input data to a geometric feature space in two example problems. The first example is curvature of surfaces, which has met renewed interest since the introduction of ubiquitous point cloud data and the maturation of the discrete differential geometry. Curvature spectra can characterize a spatial sample rather well, and provide useful features for ML purposes. The second example involves projective methods used to video stereo-signal analysis in swimming analytics. The aim is to find meaningful local geometric representations for feature generation, which also facilitate additional analysis based on geometric understanding of the model. The features are associated directly to some geometric quantity, and this makes it easier to express the geometric constraints in a natural way, as shown in the thesis. Also, the visualization and further feature generation is much easier. Third, the approach provides sound baseline methods to more traditional ML approaches, e.g. neural network methods. Fourth, most of the ML methods can utilize the geometric features presented in this work as additional features.Geometriassa käytetään perinteisesti tarkkoja malleja, jolloin datassa esiintyvät epätarkkuudet edustavat melua. Toisessa perinteessä nojataan suuren datamäärän tilastolliseen luonteeseen, jolloin geometrinen säännönmukaisuus on datan sisäsyntyinen ominaisuus, joka hahmotetaan tilastollisilla malleilla ainoastaan epäsuorasti. Tämä työ keskittyy kahteen esimerkkiin: luonnonvaroja kuvaaviin pistepilviin ja videohahmontunnistukseen. Nämä ovat todellisia ongelmia, joissa geometrinen sisältö on tavoittamattomissa raakadatan tasolla. Tämä sisältö voitaisiin jossain määrin löytää ja mallintaa koneoppimisen keinoin, esim. syväoppimisen avulla, mutta joko geometria pitää kattaa suoraan näytteistämällä tai tarvitaan neuronien lisäkerros geometrisia invariansseja varten. Geometrinen sisältö on keskeinen, kun tarvitaan suoraa avaruudellisten suureiden havainnointia, tai kun tarvitaan kuvaus geometrisesti yhtenäiseen dataesitykseen, jossa poikkeavat näytteet tai melu voidaan helposti erottaa. Tässä työssä tarkastellaan datan muuntamista geometriseen piirreavaruuteen kahden esimerkkiohjelman suhteen. Ensimmäinen esimerkki on pintakaarevuus, joka on uudelleen virinneen kiinnostuksen kohde kaikkialle saatavissa olevan datan ja diskreetin geometrian kypsymisen takia. Kaarevuusspektrit voivat luonnehtia avaruudellista kohdetta melko hyvin ja tarjota koneoppimisessa hyödyllisiä piirteitä. Toinen esimerkki koskee projektiivisia menetelmiä käytettäessä stereovideosignaalia uinnin analytiikkaan. Tavoite on löytää merkityksellisiä paikallisen geometrian esityksiä, jotka samalla mahdollistavat muun geometrian ymmärrykseen perustuvan analyysin. Piirteet liittyvät suoraan johonkin geometriseen suureeseen, ja tämä helpottaa luonnollisella tavalla geometristen rajoitteiden käsittelyä, kuten väitöstyössä osoitetaan. Myös visualisointi ja lisäpiirteiden luonti muuttuu helpommaksi. Kolmanneksi, lähestymistapa suo selkeän vertailumenetelmän perinteisemmille koneoppimisen lähestymistavoille, esim. hermoverkkomenetelmille. Neljänneksi, useimmat koneoppimismenetelmät voivat hyödyntää tässä työssä esitettyjä geometrisia piirteitä lisäämällä ne muiden piirteiden joukkoon

Feature-based hybrid inspection planning for complex mechanical parts

Globalization and emerging new powers in the manufacturing world are among many challenges, major manufacturing enterprises are facing. This resulted in increased alternatives to satisfy customers\u27 growing needs regarding products\u27 aesthetic and functional requirements. Complexity of part design and engineering specifications to satisfy such needs often require a better use of advanced and more accurate tools to achieve good quality. Inspection is a crucial manufacturing function that should be further improved to cope with such challenges. Intelligent planning for inspection of parts with complex geometric shapes and free form surfaces using contact or non-contact devices is still a major challenge. Research in segmentation and localization techniques should also enable inspection systems to utilize modern measurement technologies capable of collecting huge number of measured points. Advanced digitization tools can be classified as contact or non-contact sensors. The purpose of this thesis is to develop a hybrid inspection planning system that benefits from the advantages of both techniques. Moreover, the minimization of deviation of measured part from the original CAD model is not the only characteristic that should be considered when implementing the localization process in order to accept or reject the part; geometric tolerances must also be considered. A segmentation technique that deals directly with the individual points is a necessary step in the developed inspection system, where the output is the actual measured points, not a tessellated model as commonly implemented by current segmentation tools. The contribution of this work is three folds. First, a knowledge-based system was developed for selecting the most suitable sensor using an inspection-specific features taxonomy in form of a 3D Matrix where each cell includes the corresponding knowledge rules and generate inspection tasks. A Travel Salesperson Problem (TSP) has been applied for sequencing these hybrid inspection tasks. A novel region-based segmentation algorithm was developed which deals directly with the measured point cloud and generates sub-point clouds, each of which represents a feature to be inspected and includes the original measured points. Finally, a new tolerance-based localization algorithm was developed to verify the functional requirements and was applied and tested using form tolerance specifications. This research enhances the existing inspection planning systems for complex mechanical parts with a hybrid inspection planning model. The main benefits of the developed segmentation and tolerance-based localization algorithms are the improvement of inspection decisions in order not to reject good parts that would have otherwise been rejected due to misleading results from currently available localization techniques. The better and more accurate inspection decisions achieved will lead to less scrap, which, in turn, will reduce the product cost and improve the company potential in the market

Automatic tolerance inspection through Reverse Engineering: a segmentation technique for plastic injection moulded parts

This work studies segmentations procedures to recognise features in a Reverse Engineering (RE) application that is oriented to computer-aided tolerance inspection of injection moulding die set-up, necessary to manufacture electromechanical components. It will discuss all steps of the procedures, from the initial acquisition to the final measure data management, but specific original developments will be focused on the RE post-processing method, that should solve the problem related to the automation of the surface recognition and then of the inspection process. As it will be explained in the first two Chapters, automation of the inspection process pertains, eminently, to feature recognition after the segmentation process. This work presents a voxel-based approach with the aim of reducing the computation efforts related to tessellation and curvature analysis, with or without filtering. In fact, a voxel structure approximates the shape through parallelepipeds that include small sub-set of points. In this sense, it represents a filter, since the number of voxels is less than the total number of points, but also a local approximation of the surface, if proper fitting models are applied. Through sensitivity analysis and industrial applications, limits and perspectives of the proposed algorithms are discussed and validated in terms of accuracy and save of time. Validation case-studies are taken from real applications made in ABB Sace S.p.A., that promoted this research. Plastic injection moulding of electromechanical components has a time-consuming die set-up. It is due to the necessity of providing dies with many cavities, which during the cooling phase may present different stamping conditions, thus defects that include lengths outside their dimensional tolerance, and geometrical errors. To increase the industrial efficiency, the automation of the inspection is not only due to the automatic recognition of features but also to a computer-aided inspection protocol (path planning and inspection data management). For this reason, also these steps will be faced, as the natural framework of the thesis research activity. The work structure concerns with six chapters. In Chapter 1, an introduction to the whole procedure is presented, focusing on reasons and utilities of the application of RE techniques in industrial engineering. Chapter 2 analyses acquisition issues and methods that are related to our application, describing: (a) selected hardware; (b) adopted strategy related to the cloud of point acquisition. In Chapter 3, the proposed RE post-processing is described together with a state of art about data segmentation and surface reconstruction. Chapter 4 discusses the proposed algorithms through sensitivity studies concerning thresholds and parameters utilised in segmentation phase and surface reconstruction. Chapter 5 explains briefly the inspection workflow, PDM requirements and solution, together with a preliminary assessing of measures and their reliability. These three chapters (3, 4 and 5) report final sections, called “Discussion”, in which specific considerations are given. Finally, Chapter 6 gives examples of the proposed segmentation technique in the framework of the industrial applications, through specific case studies

Preserving the impossible: conservation of soft-sediment hominin footprint sites and strategies for three-dimensional digital data capture.

Human footprints provide some of the most publically emotive and tangible evidence of our ancestors. To the scientific community they provide evidence of stature, presence, behaviour and in the case of early hominins potential evidence with respect to the evolution of gait. While rare in the geological record the number of footprint sites has increased in recent years along with the analytical tools available for their study. Many of these sites are at risk from rapid erosion, including the Ileret footprints in northern Kenya which are second only in age to those at Laetoli (Tanzania). Unlithified, soft-sediment footprint sites such these pose a significant geoconservation challenge. In the first part of this paper conservation and preservation options are explored leading to the conclusion that to 'record and digitally rescue' provides the only viable approach. Key to such strategies is the increasing availability of three-dimensional data capture either via optical laser scanning and/or digital photogrammetry. Within the discipline there is a developing schism between those that favour one approach over the other and a requirement from geoconservationists and the scientific community for some form of objective appraisal of these alternatives is necessary. Consequently in the second part of this paper we evaluate these alternative approaches and the role they can play in a 'record and digitally rescue' conservation strategy. Using modern footprint data, digital models created via optical laser scanning are compared to those generated by state-of-the-art photogrammetry. Both methods give comparable although subtly different results. This data is evaluated alongside a review of field deployment issues to provide guidance to the community with respect to the factors which need to be considered in digital conservation of human/hominin footprints