Viewfinder: final activity report

The VIEW-FINDER project (2006-2009) is an 'Advanced Robotics' project that seeks to apply a semi-autonomous robotic system to inspect ground safety in the event of a fire. Its primary aim is to gather data (visual and chemical) in order to assist rescue personnel. A base station combines the gathered information with information retrieved from off-site sources. The project addresses key issues related to map building and reconstruction, interfacing local command information with external sources, human-robot interfaces and semi-autonomous robot navigation. The VIEW-FINDER system is a semi-autonomous; the individual robot-sensors operate autonomously within the limits of the task assigned to them, that is, they will autonomously navigate through and inspect an area. Human operators monitor their operations and send high level task requests as well as low level commands through the interface to any nodes in the entire system. The human interface has to ensure the human supervisor and human interveners are provided a reduced but good and relevant overview of the ground and the robots and human rescue workers therein

Secure Mobile Computing by Using Convolutional and Capsule Deep Neural Networks

Mobile devices are becoming smarter to satisfy modern user\u27s increasing needs better, which is achieved by equipping divers of sensors and integrating the most cutting-edge Deep Learning (DL) techniques. As a sophisticated system, it is often vulnerable to multiple attacks (side-channel attacks, neural backdoor, etc.). This dissertation proposes solutions to maintain the cyber-hygiene of the DL-Based smartphone system by exploring possible vulnerabilities and developing countermeasures. First, I actively explore possible vulnerabilities on the DL-Based smartphone system to develop proactive defense mechanisms. I discover a new side-channel attack on smartphones using the unrestricted magnetic sensor data. I demonstrate that attackers can effectively infer the Apps being used on a smartphone with an accuracy of over 80%, through training a deep Convolutional Neural Networks (CNN). Various signal processing strategies have been studied for feature extractions, including a tempogram based scheme. Moreover, by further exploiting the unrestricted motion sensor to cluster magnetometer data, the sniffing accuracy can increase to as high as 98%. To mitigate such attacks, I propose a noise injection scheme that can effectively reduce the App sniffing accuracy to only 15% and, at the same time, has a negligible effect on benign Apps. On the other hand, I leverage the DL technique to build reactive malware detection schemes. I propose an innovative approach, named CapJack, to detect in-browser malicious cryptocurrency mining activities by using the latest CapsNet technology. To the best of our knowledge, this is the first work to introduce CapsNet to the field of malware detection through system-behavioural analysis. It is particularly useful to detect malicious miners under multitasking environments where multiple applications run simultaneously. Finally, as DL itself is vulnerable to model-based attacks, I proactively explore possible attacks against the DL model. To this end, I discover a new clean label attack, named Invisible Poison, which stealthily and aggressively plants a backdoor in neural networks (NN). It converts a trigger to noise concealed inside regular images for training NN, to plant a backdoor that can be later activated by the trigger. The attack has the following distinct properties. First, it is a black-box attack, requiring zero-knowledge about the target NN model. Second, it employs \invisible poison to achieve stealthiness where the trigger is disguised as \noise that is therefore invisible to human, but at the same time, still remains significant in the feature space and thus is highly effective to poison training data

VISUAL SEMANTIC SEGMENTATION AND ITS APPLICATIONS

This dissertation addresses the difficulties of semantic segmentation when dealing with an extensive collection of images and 3D point clouds. Due to the ubiquity of digital cameras that help capture the world around us, as well as the advanced scanning techniques that are able to record 3D replicas of real cities, the sheer amount of visual data available presents many opportunities for both academic research and industrial applications. But the mere quantity of data also poses a tremendous challenge. In particular, the problem of distilling useful information from such a large repository of visual data has attracted ongoing interests in the fields of computer vision and data mining. Structural Semantics are fundamental to understanding both natural and man-made objects. Buildings, for example, are like languages in that they are made up of repeated structures or patterns that can be captured in images. In order to find these recurring patterns in images, I present an unsupervised frequent visual pattern mining approach that goes beyond co-location to identify spatially coherent visual patterns, regardless of their shape, size, locations and orientation. First, my approach categorizes visual items from scale-invariant image primitives with similar appearance using a suite of polynomial-time algorithms that have been designed to identify consistent structural associations among visual items, representing frequent visual patterns. After detecting repetitive image patterns, I use unsupervised and automatic segmentation of the identified patterns to generate more semantically meaningful representations. The underlying assumption is that pixels capturing the same portion of image patterns are visually consistent, while pixels that come from different backdrops are usually inconsistent. I further extend this approach to perform automatic segmentation of foreground objects from an Internet photo collection of landmark locations. New scanning technologies have successfully advanced the digital acquisition of large-scale urban landscapes. In addressing semantic segmentation and reconstruction of this data using LiDAR point clouds and geo-registered images of large-scale residential areas, I develop a complete system that simultaneously uses classification and segmentation methods to first identify different object categories and then apply category-specific reconstruction techniques to create visually pleasing and complete scene models

A novel monitoring system for fall detection in older people

Indexación: Scopus.This work was supported in part by CORFO - CENS 16CTTS-66390 through the National Center on Health Information Systems, in part by the National Commission for Scientific and Technological Research (CONICYT) through the Program STIC-AMSUD 17STIC-03: ‘‘MONITORing for ehealth," FONDEF ID16I10449 ‘‘Sistema inteligente para la gestión y análisis de la dotación de camas en la red asistencial del sector público’’, and in part by MEC80170097 ‘‘Red de colaboración científica entre universidades nacionales e internacionales para la estructuración del doctorado y magister en informática médica en la Universidad de Valparaíso’’. The work of V. H. C. De Albuquerque was supported by the Brazilian National Council for Research and Development (CNPq), under Grant 304315/2017-6.Each year, more than 30% of people over 65 years-old suffer some fall. Unfortunately, this can generate physical and psychological damage, especially if they live alone and they are unable to get help. In this field, several studies have been performed aiming to alert potential falls of the older people by using different types of sensors and algorithms. In this paper, we present a novel non-invasive monitoring system for fall detection in older people who live alone. Our proposal is using very-low-resolution thermal sensors for classifying a fall and then alerting to the care staff. Also, we analyze the performance of three recurrent neural networks for fall detections: Long short-term memory (LSTM), gated recurrent unit, and Bi-LSTM. As many learning algorithms, we have performed a training phase using different test subjects. After several tests, we can observe that the Bi-LSTM approach overcome the others techniques reaching a 93% of accuracy in fall detection. We believe that the bidirectional way of the Bi-LSTM algorithm gives excellent results because the use of their data is influenced by prior and new information, which compares to LSTM and GRU. Information obtained using this system did not compromise the user's privacy, which constitutes an additional advantage of this alternative. © 2013 IEEE.https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=842305

Real-time spatial modeling to detect and track resources on construction sites

For more than 10 years the U.S. construction industry has experienced over 1,000 fatalities annually. Many fatalities may have been prevented had the individuals and equipment involved been more aware of and alert to the physical state of the environment around them. Awareness may be improved by automatic 3D (three-dimensional) sensing and modeling of the job site environment in real-time. Existing 3D modeling approaches based on range scanning techniques are capable of modeling static objects only, and thus cannot model in real-time dynamic objects in an environment comprised of moving humans, equipment, and materials. Emerging prototype 3D video range cameras offer another alternative by facilitating affordable, wide field of view, automated static and dynamic object detection and tracking at frame rates better than 1Hz (real-time). This dissertation presents an imperical work and methodology to rapidly create a spatial model of construction sites and in particular to detect, model, and track the position, dimension, direction, and velocity of static and moving project resources in real-time, based on range data obtained from a three-dimensional video range camera in a static or moving position. Existing construction site 3D modeling approaches based on optical range sensing technologies (laser scanners, rangefinders, etc.) and 3D modeling approaches (dense, sparse, etc.) that offered potential solutions for this research are reviewed. The choice of an emerging sensing tool and preliminary experiments with this prototype sensing technology are discussed. These findings led to the development of a range data processing algorithm based on three-dimensional occupancy grids which is demonstrated in detail. Testing and validation of the proposed algorithms have been conducted to quantify the performance of sensor and algorithm through extensive experimentation involving static and moving objects. Experiments in indoor laboratory and outdoor construction environments have been conducted with construction resources such as humans, equipment, materials, or structures to verify the accuracy of the occupancy grid modeling approach. Results show that modeling objects and measuring their position, dimension, direction, and speed had an accuracy level compatible to the requirements of active safety features for construction. Results demonstrate that video rate 3D data acquisition and analysis of construction environments can support effective detection, tracking, and convex hull modeling of objects. Exploiting rapidly generated three-dimensional models for improved visualization, communications, and process control has inherent value, broad application, and potential impact, e.g. as-built vs. as-planned comparison, condition assessment, maintenance, operations, and construction activities control. In combination with effective management practices, this sensing approach has the potential to assist equipment operators to avoid incidents that result in reduce human injury, death, or collateral damage on construction sites.Civil, Architectural, and Environmental Engineerin

Integrating virtual reality and GIS tools for geological mapping, data collection and analysis: an example from Metaxa Mine, Santorini (Greece)

In the present work we highlight the effectiveness of integrating different techniques and tools for better surveying, mapping and collecting data in volcanic areas. We use an Immersive Virtual Reality (IVR) approach for data collection, integrated with Geographic Information System (GIS) analysis in a well-known volcanological site in Santorini (Metaxa mine), a site where volcanic processes influenced the island’s industrial development, especially with regard to pumice mining. Specifically, we have focused on: (i) three-dimensional (3D) high-resolution IVR scenario building, based on Structure from Motion photogrammetry (SfM) modeling; (ii) subsequent geological survey, mapping and data collection using IVR; (iii) data analysis, e.g., calculation of extracted volumes, as well as production of new maps in a GIS environment using input data directly from the IVR survey; and finally, (iv) presentation of new outcomes that highlight the importance of the Metaxa Mine as a key geological and volcanological geosite