Deep Learning in Graph Domains for Sensorised Environments

As our society moves swiftly towards an era where technology seamlessly integrates into our daily lives, our homes and cities are becoming increasingly sensorised. This change is fueled by advancements in artificial intelligence that facilitate harnessing the potential of smart environments. The main focus of this thesis is to investigate how Graph Neural Networks (GNNs) can be effectively applied to these environments, with a focus on those where humans and robots share the space. In these scenarios, integrating and exploiting data from multiple sources and analysing interactions between individuals, objects, sensors and robots is paramount. As the literature shows, GNNs have advantageous properties to process this kind of data when compared to more established deep learning approaches. This thesis presents a range of methods and applications in sensorised environments that leverage GNNs’ properties. The main contributions span applications in three main fields: human-aware robot navigation, human pose estimation, and the generation of traffic images. For human-aware navigation, this thesis proposes a model capable of estimating the level of discomfort caused by a robot’s presence among people and objects, considering not only the entities themselves but also the interactions happening. This model is later improved to yield discomfort maps that can be used as cost maps for motion planning. In the domain of human pose estimation, two different solutions are presented: a model capable of estimating the position and orientation of the people in the environment, and a multi-camera and multi-person 3D human full pose estimator. This last model, which does not require a labelled dataset for training, can be used for tracking people and feed their poses into the aforementioned cost map generator, as seen in the experimentation of this thesis. These works exhibit superior results in terms of precision, accuracy, and time efficiency when compared to similar state-of-the-art works. Finally, in the field of image generation, the thesis explores an application within the context of smart cities: generating realistic traffic images conditioned with graphs. This work leverages the strengths of GNNs when working with semantic data. The model can generate realistic images based on the properties of the items expected in them –namely their position, size and colour– and global properties such as the time of day. GNNs can be time-inefficient due to the added complexity of dealing with heterogeneously structured data. Consequently, the success of the applications presented in this thesis is the result of the effective integration of this networks, often in conjunction with other well-known approaches. One notable example is the fusion of convolutional networks with GNNs, which in this thesis leads to more efficient image generation when compared to pure GNN architectures. These methods constitute the central contribution of this thesis, as they allow GNNs to fully exploit their potential while mitigating inefficiencies

Multi-person 3D pose estimation from unlabelled data

Its numerous applications make multi-human 3D pose estimation a remarkably impactful area of research. Nevertheless, assuming a multiple-view system composed of several regular RGB cameras, 3D multi-pose estimation presents several challenges. First of all, each person must be uniquely identified in the different views to separate the 2D information provided by the cameras. Secondly, the 3D pose estimation process from the multi-view 2D information of each person must be robust against noise and potential occlusions in the scenario. In this work, we address these two challenges with the help of deep learning. Specifically, we present a model based on Graph Neural Networks capable of predicting the cross-view correspondence of the people in the scenario along with a Multilayer Perceptron that takes the 2D points to yield the 3D poses of each person. These two models are trained in a self-supervised manner, thus avoiding the need for large datasets with 3D annotations

Multi-camera Torso Pose Estimation using Graph Neural Networks

Estimating the location and orientation of humans is an essential skill for service and assistive robots. To achieve a reliable estimation in a wide area such as an apartment, multiple RGBD cameras are frequently used. Firstly, these setups are relatively expensive. Secondly, they seldom perform an effective data fusion using the multiple camera sources at an early stage of the processing pipeline. Occlusions and partial views make this second point very relevant in these scenarios. The proposal presented in this paper makes use of graph neural networks to merge the information acquired from multiple camera sources, achieving a mean absolute error below 125 mm for the location and 10 degrees for the orientation using low-resolution RGB images. The experiments, conducted in an apartment with three cameras, benchmarked two different graph neural network implementations and a third architecture based on fully connected layers. The software used has been released as open-source in a public repository

A model to support collective reasoning: Formalization, analysis and computational assessment

Inspired by e-participation systems, in this paper we propose a new model to represent human debates and methods to obtain collective conclusions from them. This model overcomes drawbacks of existing approaches by allowing users to introduce new pieces of information into the discussion, to relate them to existing pieces, and also to express their opinion on the pieces proposed by other users. In addition, our model does not assume that users' opinions are rational in order to extract information from it, an assumption that significantly limits current approaches. Instead, we define a weaker notion of rationality that characterises coherent opinions, and we consider different scenarios based on the coherence of individual opinions and the level of consensus that users have on the debate structure. Considering these two factors, we analyse the outcomes of different opinion aggregation functions that compute a collective decision based on the individual opinions and the debate structure. In particular, we demonstrate that aggregated opinions can be coherent even if there is a lack of consensus and individual opinions are not coherent. We conclude our analysis with a computational evaluation demonstrating that collective opinions can be computed efficiently for real-sized debates

Estrés postraumático y congniciones irracionales en damnificados por el invierno Santa Marta Colombia 2013

La presente investigación busca la existencia de estrés postraumático y cogniciones irracionales en los damnificados por el invierno en el barrio las Malvinas de la ciudad de Santa Marta Colombia. Aplicando así una encuesta de manera aleatoria a una población de cuarenta y cuatro personas mayores de dieciocho años de edad habitantes del mismo barrio; en la que se utilizó la escala de síntomas de Davidson (1997) escala del trauma; la cual fue elaborada para evaluar y medir la frecuencia y severidad de los síntomas del trastorno por estrés postraumático en sujetos que han sufrido un evento estresante. Y el inventario de cogniciones postraumáticas Foa et al (1999), Tiene como fin evaluar pensamientos y creencias relacionados con el trauma esta consta de tres subescalas que miden cogniciones negativas acerca de sí mismo, cogniciones negativas acerca del mundo y autoculpa. Los resultados indican en la evaluación por criterios para el diagnóstico del Trastorno de Estrés Postraumático (TEPT), el 7,01% (4) de la población, no presentaba este trastorno, el 81,37%(35) lo presenta de una manera leve-moderada, en cambio el 11,62%(5) de la población, lo presenta de una manera severa. Al analizar los resultados del Inventario de Cogniciones Irracionales Postraumáticas por criterios, se evidencia que el mayor porcentaje se encuentra en el criterio de Cogniciones Negativas acerca del Mundo

Using Magentix2 in Smart-Home Environments

[EN] In this paper, we present the application of a multi-agent platform Magentix2 for the development of MAS in smart-homes. Specificallly, the use of Magentix2 (http://gti-ia.upv.es/sma/tools/magentix2/index.php) platform facilitates the management of the multiple occupancy in smart living spaces. Virtual organizations provide the possibility of defining a set of norms and roles that facilitate the regulation and control of the actions that can be carried out by the internal and external agents depending on their profile. We illustrate the applicability of our proposal with a set of scenarios. © Springer International Publishing Switzerland 2015.This work is supported by the Spanish government grants CONSOLIDER INGENIO 2010 CSD2007-00022, MINECO/FEDER TIN2012-36586-C03-01, TIN2011-27652-C03-01, and SP2014800.Valero Cubas, S.; Del Val Noguera, E.; Alemany Bordera, J.; Botti, V. (2015). Using Magentix2 in Smart-Home Environments. En 10th International Conference on Soft Computing Models in Industrial and Environmental Applications. Springer Verlag. 27-37. https://doi.org/10.1007/978-3-319-19719-7_3S2737Bajo J, Fraile JA, Pérez-Lancho B, Corchado JM (2010) The thomas architecture in home care scenarios: a case study. Expert Syst Appl 37(5):3986–3999Cetina C, Giner P, Fons J, Pelechano V (2009) Autonomic computing through reuse of variability models at runtime: The case of smart homes. Computer 42(10):37–43Cook DJ (2009) Multi-agent smart environments. J Ambient Intell Smart Environ 1(1):51–55Crandall AS, Cook DJ (2010) Using a hidden markov model for resident identification. In: 6th international conference on intelligent environments, pp 74–79. IEEECriado N, Argente E, Botti V (2013) THOMAS: an agent platform for supporting normative multi-agent systems. J Logic Comput 23(2):309–333Davidoff S, Lee MK, Zimmerman J, Dey A (2006) Socially-aware requirements for a smart home. In: Proceedings of the international symposium on intelligent, environments, pp 41–44Grupo de Tecnología Informática e Inteligencia Artificial (GTI-IA) (2015). http://www.gti-ia.upv.es/sma/tools/magentix2/archivos/Magentix2UserManualv2.1.0.pdf . Magentix2 User’s Manual v2.0Loseto G, Scioscia F, Ruta M, di Sciascio E (2012) Semantic-based smart homes: a multi-agent approach. In: 13th Workshop on objects and agents (WOA 2012), vol 892, pp 49–55Rodriguez S, Julián V, Bajo J, Carrascosa C, Botti V, Corchado JM (2011) Agent-based virtual organization architecture. Eng Appl Artif Intell 24(5):895–910Rodríguez S, Paz JFD, Villarrubia G, Zato C, Bajo J, Corchado JM (2015) Multi-agent information fusion system to manage data from a WSN in a residential home. Inf Fusion 23:43–57Such JM, Garca-Fornes A, Espinosa A, Bellver J (2012) Magentix2: a Privacy-enhancing Agent Platform. Eng Appl Artif IntellSun Q, Yu W, Kochurov N, Hao Q, Hu F (2013) A multi-agent-based intelligent sensor and actuator network design for smart house and home automation. J Sens Actuator Netw 2(3):557–588Val E, Criado N, Rebollo M, Argente E, Julian V (2009) Service-oriented framework for virtual organizations. 1:108–114Wu C-L, Liao C-F, Fu L-C (2007) Service-oriented smart-home architecture based on osgi and mobile-agent technology. IEEE Trans Syst Man Cybern Part C Appl Rev 37(2):193–205Yin J, Yang Q, Shen D, Li Z-N (2008) Activity recognition via user-trace segmentation. ACM Trans Sens Netw (TOSN) 4(4):1

Multi-person 3D pose estimation from unlabelled data

Its numerous applications make multi-human 3D pose estimation a remarkably impactful area of research. Nevertheless, it presents several challenges, especially when approached using multiple views and regular RGB cameras as the only input. First, each person must be uniquely identified in the different views. Secondly, it must be robust to noise, partial occlusions, and views where a person may not be detected. Thirdly, many pose estimation approaches rely on environment-specific annotated datasets that are frequently prohibitively expensive and/or require specialised hardware. Specifically, this is the first multi-camera, multi-person data-driven approach that does not require an annotated dataset. In this work, we address these three challenges with the help of self-supervised learning. In particular, we present a three-staged pipeline and a rigorous evaluation providing evidence that our approach performs faster than other state-of-the-art algorithms, with comparable accuracy, and most importantly, does not require annotated datasets. The pipeline is composed of a 2D skeleton detection step, followed by a Graph Neural Network to estimate cross-view correspondences of the people in the scenario, and a Multi-Layer Perceptron that transforms the 2D information into 3D pose estimations. Our proposal comprises the last two steps, and it is compatible with any 2D skeleton detector as input. These two models are trained in a self-supervised manner, thus avoiding the need for datasets annotated with 3D ground-truth poses

Data fusion uncertainty-enabled methods to map street-scale hourly NO2 in Barcelona: a case study with CALIOPE-Urban v1.0

Comprehensive monitoring of NO2 exceedances is imperative for protecting human health, especially in urban areas with traffic. However, an accurate spatial characterization of the exceedances is challenging due to the typically low density of air quality monitoring stations and the inherent uncertainties in urban air quality models. We study how observational data from different sources and timescales can be combined with a dispersion air quality model to obtain bias-corrected NO2 hourly maps at the street scale. We present a kriging-based data fusion workflow that merges dispersion model output with continuous hourly observations and uses a machine-learning-based land use regression (LUR) model constrained with past short intensive passive dosimeter campaign measurements. While the hourly observations allow the bias adjustment of the temporal variability in the dispersion model, the microscale LUR model adds information on the NO2 spatial patterns. Our method includes an uncertainty calculation based on the estimated error variance of the universal kriging technique, which is subsequently used to produce urban maps of probability of exceeding the 200 µg m−3 hourly and the 40 µg m−3 annual NO2 average limits. We assess the statistical performance of this approach in the city of Barcelona for the year 2019. Our results show that simply merging the monitoring stations with the model output already significantly increases the correlation coefficient (r) by +29 % and decreases the root mean square error (RMSE) by −32 %. When adding the time-invariant microscale LUR model in the data fusion workflow, the improvement is even more remarkable, with +46 % and −48 % for the r and RMSE, respectively. Our work highlights the usefulness of high-resolution spatial information in data fusion methods to better estimate exceedances at the street scale.We have received support from the Barcelona City Council through the UncertAIR project (ID 22S09501-001; Recerca Jove i emergent 2022). This research has been supported by the Ministerio de Ciencia e Innovación through the BROWNING project (grant no. RTI2018-099894-BI00), the Agencia Estatal de Investigación as part of the VITALISE project (grant no. PID2019-108086RA-I00) and the MITIGATE project (grant no. PID2020-116324RA695 I00), the H2020 Marie Skłodowska-Curie Actions (grant no. H2020-MSCA-COFUND-2016-754433), the AXA Research Fund, and the Barcelona Supercomputing Center (grant nos. RES-AECT-2021-1-0027 and RES-AECT-2021-2-0001).Peer ReviewedPostprint (author's final draft

A Toolkit to Generate Social Navigation Datasets

Social navigation datasets are necessary to assess social navigation algorithms and train machine learning algorithms. Most of the currently available datasets target pedestrians’ movements as a pattern to be replicated by robots. It can be argued that one of the main reasons for this to happen is that compiling datasets where real robots are manually controlled, as they would be expected to behave when moving, is a very resource-intensive task. Another aspect that is often missing in datasets is symbolic information that could be relevant, such as human activities, relationships or interactions. Unfortunately, the available datasets targeting robots and supporting symbolic information are restricted to static scenes. This paper argues that simulation can be used to gather social navigation data in an effective and cost-efficient way and presents a toolkit for this purpose. A use case studying the application of graph neural networks to create learned control policies using supervised learning is presented as an example of how it can be used