Developing Advanced Photogrammetric Methods for Automated Rockfall Monitoring

[eng] In recent years, photogrammetric models have become a widely used tool in the field of geosciences thanks to their ability to reproduce natural surfaces. As an alternative to other systems such as LiDAR (Light Detection and Ranging), photogrammetry makes it possible to obtain 3D points clouds at a lower cost and with a lower learning curve. This combination has allowed the democratisation of this 3D model creation strategy. On the other hand, rockfalls are one of the geological phenomena that represent a risk for society. It is the most common natural phenomenon in mountainous areas and, given its great speed, its hazard is very high. This doctoral thesis deals with the creation of photogrammetric systems and processing algorithms for the automatic monitoring of rockfalls. To this end, 3 fixed camera photogrammetric systems were designed and installed in 2 study areas. In addition, 3 different workflows have been developed, two of which are aimed at obtaining comparisons of higher quality using photogrammetric models and the other focused on automating the entire monitoring process with the aim of obtaining automatic monitoring systems of low temporal frequency. The photogrammetric RasPi system has been designed and installed in the study area of Puigcercós (Catalonia). This very low-cost system has been designed using Raspberry cameras. Despite being a very low-cost and low-resolution system, the results obtained demonstrate its ability to identify rockfalls and pre-failure deformation. The HRCam photogrammetric system has also been designed and installed in the Puigcercós study area. This system uses commercial cameras and more complex control systems. With this system, higher quality models have been obtained that enable better monitoring of rockfalls. Finally, the DSLR system has been designed similarly to the HRCam system but has been installed in a real risk area in the Tajo de San Pedro in the Alhambra (Andalusia). This system has been used to constantly monitor the rockfalls affecting this escarpment. In order to obtain 3D comparisons with the highest possible quality, two workflows have been developed. The first, called PCStacking, consists of stacking 3D models in order to calculate the median of the Z coordinates of each point to generate a new averaged point cloud. This thesis shows the application of the algorithm both with ad hoc created synthetic point clouds and with real point clouds. In both cases, the 25th and 75th percentile errors of the 3D comparisons were reduced from 3.2 cm to 1.4 cm in synthetic tests and from 1.5 cm to 0.5 cm in real conditions. The second workflow that has been developed is called MEMI (Multi-Epoch and Multi-Imagery). This workflow is capable of obtaining photogrammetric comparisons with a higher quality than those obtained with the classical workflow. The redundant use of images from the two periods to be compared reduces the error to a factor of 2 compared to the classical approach, yielding a standard deviation of the comparison of 3D models of 1.5 cm. Finally, the last workflow presented in this thesis is an update and an automation of the method for detecting rockfalls from point-clouds carried out by the RISKNAT research group. The update has been carried out with two objectives in mind. The first is to transfer the entire working method to free licence (both language and programming), and the second is to include in the processing the new algorithms and improvements that have recently been developed. The automation of the method has been performed to cope with the large amount of data generated by photogrammetric systems. It consists of automating all the processes, which means that everything from the capture of the image in the field to the obtention of the rockfalls is performed automatically. This automation poses important challenges, which, although not completely solved, are addressed in this thesis. Thanks to the creation of photogrammetric systems, 3D model improvement algorithms and automation of the rockfall identification workflow, this doctoral thesis presents a solid and innovative proposal in the field of low-cost automatic monitoring. The creation of these systems and algorithms constitutes a further step in the unimpeded expansion of monitoring and warning systems, whose ultimate goal is to enable us to live in a safer world and to build more resilient societies to deal with geological hazards.[cat] En els darrers anys, els models fotogramètrics s’han convertit en una eina molt utilitzada en l’àmbit de les geociències gràcies a la seva capacitat per reproduir superfícies naturals. Com a alternativa a altres sistemes com el LiDAR (Light Detection and Ranging), la fotogrametria permet obtenir núvols de punts 3D a un cost més baix i amb una corba d’aprenentatge menor. Per altra banda, els despreniments de roca són un dels fenòmens geològics que representen un risc per al conjunt de la societat. Aquesta tesi doctoral aborda la creació de sistemes fotogramètrics i algoritmes de processat per al monitoratge automàtic de despreniments de roca. Per una banda, s’ha dissenyat un sistema fotogramètric de molt baix cost fent servir càmeres Raspberry Pi, anomenat RasPi System, instal·lat a la zona d’estudi de Puigcercós (Catalunya). Per altra banda, s’ha dissenyat un sistema fotogramètric d’alta resolució anomenat HRCam també instal·lat a la zona d’estudi de Puigcercós. Finalment, s’ha dissenyat un tercer sistema fotogramètric de manera similar al sistema HRCam anomenat DSLR, instal·lat en una zona de risc real al Tajo de San Pedro de l’Alhambra (Andalusia). Per obtenir comparacions 3D amb la màxima qualitat possible, s’han desenvolupat dos fluxos de treball. El primer, anomenat PCStacking consisteix a realitzar un apilament de models 3D per tal de calcular la mediana de les coordenades Z de cada punt. El segon flux de treball que s’ha desenvolupat s’anomena MEMI (Multi-Epoch and Multi-Imagery). Aquest flux de treball és capaç d’obtenir comparacions fotogramètriques amb una qualitat superior a les que s’obtenen amb el flux de treball clàssic. Finalment, el darrer flux de treball que es presenta en aquesta tesi és una actualització i una automatització del mètode de detecció de despreniments de roca del grup de recerca RISKNAT. L’actualització s’ha dut a terme perseguint dos objectius. El primer, traspassar tot el mètode de treball a llicència lliure (tant llenguatge com programari) i el segon, incloure els nous algoritmes i millores desenvolupats en aquesta tesi en el processat fotogramètric Gràcies a la creació dels sistemes fotogramètrics, algoritmes de millora de models 3D i l’automatització en la identificació de despreniments aquesta tesi doctoral presenta una proposta sòlida i innovadora en el camp del monitoratge automàtic de baix cost. La creació d’aquests sistemes i algoritmes representen un avenç important en l’expansió dels sistemes de monitoratge i alerta que tenen com a objectiu final permetre'ns viure en un món més segur i construir societats més resilients enfront dels riscos geològics

Smart Parking System Using Color QR Code

In today’s world, parking area constitutes nearly most of traffic congestion is caused by vehicles cruising around their destination and looking for a place to park. Due to this reason many day-to-day activities are affected such as waste of time, fuel wastage, frustration to drivers, theft fear, pollution etc. These factors motivated to pave a new method for smart parking system. In this method the detection is reliable, even when tests are performed using images captured from a different viewpoint. It also provides to design a highly reliable & compatible image segmentation measures for parking slot identification system and a user key driven data base measures to detect the vehicle using theft alarm system

Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications

We present Chameleon, a novel hybrid (mixed-protocol) framework for secure function evaluation (SFE) which enables two parties to jointly compute a function without disclosing their private inputs. Chameleon combines the best aspects of generic SFE protocols with the ones that are based upon additive secret sharing. In particular, the framework performs linear operations in the ring $\mathbb{Z}_{2^l}$ using additively secret shared values and nonlinear operations using Yao's Garbled Circuits or the Goldreich-Micali-Wigderson protocol. Chameleon departs from the common assumption of additive or linear secret sharing models where three or more parties need to communicate in the online phase: the framework allows two parties with private inputs to communicate in the online phase under the assumption of a third node generating correlated randomness in an offline phase. Almost all of the heavy cryptographic operations are precomputed in an offline phase which substantially reduces the communication overhead. Chameleon is both scalable and significantly more efficient than the ABY framework (NDSS'15) it is based on. Our framework supports signed fixed-point numbers. In particular, Chameleon's vector dot product of signed fixed-point numbers improves the efficiency of mining and classification of encrypted data for algorithms based upon heavy matrix multiplications. Our evaluation of Chameleon on a 5 layer convolutional deep neural network shows 133x and 4.2x faster executions than Microsoft CryptoNets (ICML'16) and MiniONN (CCS'17), respectively

Contribuciones a la estimación de la pose de la cámara en aplicaciones industriales de realidad aumentada

Augmented Reality (AR) aims to complement the visual perception of the user environment superimposing virtual elements. The main challenge of this technology is to combine the virtual and real world in a precise and natural way. To carry out this goal, estimating the user position and orientation in both worlds at all times is a crucial task. Currently, there are numerous techniques and algorithms developed for camera pose estimation. However, the use of synthetic square markers has become the fastest, most robust and simplest solution in these cases. In this scope, a big number of marker detection systems have been developed. Nevertheless, most of them presents some limitations, (1) their unattractive and non-customizable visual appearance prevent their use in industrial products and (2) the detection rate is drastically reduced in presence of noise, blurring and occlusions. In this doctoral dissertation the above-mentioned limitations are addressed. In first place, a comparison has been made between the different marker detection systems currently available in the literature, emphasizing the limitations of each. Secondly, a novel approach to design, detect and track customized markers capable of easily adapting to the visual limitations of commercial products has been developed. In third place, a method that combines the detection of black and white square markers with keypoints and contours has been implemented to estimate the camera position in AR applications. The main motivation of this work is to offer a versatile alternative (based on contours and keypoints) in cases where, due to noise, blurring or occlusions, it is not possible to identify markers in the images. Finally, a method for reconstruction and semantic segmentation of 3D objects using square markers in photogrammetry processes has been presented.La Realidad Aumentada (AR) tiene como objetivo complementar la percepción visual del entorno circunstante al usuario mediante la superposición de elementos virtuales. El principal reto de dicha tecnología se basa en fusionar, de forma precisa y natural, el mundo virtual con el mundo real. Para llevar a cabo dicha tarea, es de vital importancia conocer en todo momento tanto la posición, así como la orientación del usuario en ambos mundos. Actualmente, existen un gran número de técnicas de estimación de pose. No obstante, el uso de marcadores sintéticos cuadrados se ha convertido en la solución más rápida, robusta y sencilla utilizada en estos casos. En este ámbito de estudio, existen un gran número de sistemas de detección de marcadores ampliamente extendidos. Sin embargo, su uso presenta ciertas limitaciones, (1) su aspecto visual, poco atractivo y nada customizable impiden su uso en ciertos productos industriales en donde la personalización comercial es un aspecto crucial y (2) la tasa de detección se ve duramente decrementada ante la presencia de ruido, desenfoques y oclusiones Esta tesis doctoral se ocupa de las limitaciones anteriormente mencionadas. En primer lugar, se ha realizado una comparativa entre los diferentes sistemas de detección de marcadores actualmente en uso, enfatizando las limitaciones de cada uno. En segundo lugar, se ha desarrollado un novedoso enfoque para diseñar, detectar y trackear marcadores personalizados capaces de adaptarse fácilmente a las limitaciones visuales de productos comerciales. En tercer lugar, se ha implementado un método que combina la detección de marcadores cuadrados blancos y negros con keypoints y contornos, para estimar de la posición de la cámara en aplicaciones AR. La principal motivación de este trabajo se basa en ofrecer una alternativa versátil (basada en contornos y keypoints) en aquellos casos donde, por motivos de ruido, desenfoques u oclusiones no sea posible identificar marcadores en las imágenes. Por último, se ha desarrollado un método de reconstrucción y segmentación semántica de objetos 3D utilizando marcadores cuadrados en procesos de fotogrametría

On-Demand Monitoring of Construction Projects through a Game-Like Hybrid Application of BIM and Machine Learning

While unavoidable, inspections, progress monitoring, and comparing as-planned with as-built conditions in construction projects do not readily add tangible intrinsic value to the end-users. In large-scale construction projects, the process of monitoring the implementation of every single part of buildings and reflecting them on the BIM models can become highly labour intensive and error-prone, due to the vast amount of data produced in the form of schedules, reports and photo logs. In order to address the mentioned methodological and technical gap, this paper presents a framework and a proof of concept prototype for on-demand automated simulation of construction projects, integrating some cutting edge IT solutions, namely image processing, machine learning, BIM and Virtual Reality. This study utilised the Unity game engine to integrate data from the original BIM models and the as-built images, which were processed via various computer vision techniques. These methods include object recognition and semantic segmentation for identifying different structural elements through supervised training in order to superimpose the real world images on the as-planned model. The proposed framework leads to an automated update of the 3D virtual environment with states of the construction site. This framework empowers project managers and stockholders with an advanced decision-making tool, highlighting the inconsistencies in an effective manner. This paper contributes to body knowledge by providing a technical exemplar for the integration of ML and image processing approaches with immersive and interactive BIM interfaces, the algorithms and program codes of which can help replicability of these approaches by other scholars

Symmetry-Adapted Machine Learning for Information Security

Symmetry-adapted machine learning has shown encouraging ability to mitigate the security risks in information and communication technology (ICT) systems. It is a subset of artificial intelligence (AI) that relies on the principles of processing future events by learning past events or historical data. The autonomous nature of symmetry-adapted machine learning supports effective data processing and analysis for security detection in ICT systems without the interference of human authorities. Many industries are developing machine-learning-adapted solutions to support security for smart hardware, distributed computing, and the cloud. In our Special Issue book, we focus on the deployment of symmetry-adapted machine learning for information security in various application areas. This security approach can support effective methods to handle the dynamic nature of security attacks by extraction and analysis of data to identify hidden patterns of data. The main topics of this Issue include malware classification, an intrusion detection system, image watermarking, color image watermarking, battlefield target aggregation behavior recognition model, IP camera, Internet of Things (IoT) security, service function chain, indoor positioning system, and crypto-analysis

Recent Advances in Indoor Localization Systems and Technologies

Despite the enormous technical progress seen in the past few years, the maturity of indoor localization technologies has not yet reached the level of GNSS solutions. The 23 selected papers in this book present the recent advances and new developments in indoor localization systems and technologies, propose novel or improved methods with increased performance, provide insight into various aspects of quality control, and also introduce some unorthodox positioning methods