Sharing Human-Generated Observations by Integrating HMI and the Semantic Sensor Web

Current “Internet of Things” concepts point to a future where connected objects gather meaningful information about their environment and share it with other objects and people. In particular, objects embedding Human Machine Interaction (HMI), such as mobile devices and, increasingly, connected vehicles, home appliances, urban interactive infrastructures, etc., may not only be conceived as sources of sensor information, but, through interaction with their users, they can also produce highly valuable context-aware human-generated observations. We believe that the great promise offered by combining and sharing all of the different sources of information available can be realized through the integration of HMI and Semantic Sensor Web technologies. This paper presents a technological framework that harmonizes two of the most influential HMI and Sensor Web initiatives: the W3C’s Multimodal Architecture and Interfaces (MMI) and the Open Geospatial Consortium (OGC) Sensor Web Enablement (SWE) with its semantic extension, respectively. Although the proposed framework is general enough to be applied in a variety of connected objects integrating HMI, a particular development is presented for a connected car scenario where drivers’ observations about the traffic or their environment are shared across the Semantic Sensor Web. For implementation and evaluation purposes an on-board OSGi (Open Services Gateway Initiative) architecture was built, integrating several available HMI, Sensor Web and Semantic Web technologies. A technical performance test and a conceptual validation of the scenario with potential users are reported, with results suggesting the approach is soun

Context Aware Computing for The Internet of Things: A Survey

As we are moving towards the Internet of Things (IoT), the number of sensors deployed around the world is growing at a rapid pace. Market research has shown a significant growth of sensor deployments over the past decade and has predicted a significant increment of the growth rate in the future. These sensors continuously generate enormous amounts of data. However, in order to add value to raw sensor data we need to understand it. Collection, modelling, reasoning, and distribution of context in relation to sensor data plays critical role in this challenge. Context-aware computing has proven to be successful in understanding sensor data. In this paper, we survey context awareness from an IoT perspective. We present the necessary background by introducing the IoT paradigm and context-aware fundamentals at the beginning. Then we provide an in-depth analysis of context life cycle. We evaluate a subset of projects (50) which represent the majority of research and commercial solutions proposed in the field of context-aware computing conducted over the last decade (2001-2011) based on our own taxonomy. Finally, based on our evaluation, we highlight the lessons to be learnt from the past and some possible directions for future research. The survey addresses a broad range of techniques, methods, models, functionalities, systems, applications, and middleware solutions related to context awareness and IoT. Our goal is not only to analyse, compare and consolidate past research work but also to appreciate their findings and discuss their applicability towards the IoT.Comment: IEEE Communications Surveys & Tutorials Journal, 201

Ontology-based specific and exhaustive user profiles for constraint information fusion for multi-agents

Intelligent agents are an advanced technology utilized in Web Intelligence. When searching information from a distributed Web environment, information is retrieved by multi-agents on the client site and fused on the broker site. The current information fusion techniques rely on cooperation of agents to provide statistics. Such techniques are computationally expensive and unrealistic in the real world. In this paper, we introduce a model that uses a world ontology constructed from the Dewey Decimal Classification to acquire user profiles. By search using specific and exhaustive user profiles, information fusion techniques no longer rely on the statistics provided by agents. The model has been successfully evaluated using the large INEX data set simulating the distributed Web environment

Middleware Technologies for Cloud of Things - a survey

The next wave of communication and applications rely on the new services provided by Internet of Things which is becoming an important aspect in human and machines future. The IoT services are a key solution for providing smart environments in homes, buildings and cities. In the era of a massive number of connected things and objects with a high grow rate, several challenges have been raised such as management, aggregation and storage for big produced data. In order to tackle some of these issues, cloud computing emerged to IoT as Cloud of Things (CoT) which provides virtually unlimited cloud services to enhance the large scale IoT platforms. There are several factors to be considered in design and implementation of a CoT platform. One of the most important and challenging problems is the heterogeneity of different objects. This problem can be addressed by deploying suitable "Middleware". Middleware sits between things and applications that make a reliable platform for communication among things with different interfaces, operating systems, and architectures. The main aim of this paper is to study the middleware technologies for CoT. Toward this end, we first present the main features and characteristics of middlewares. Next we study different architecture styles and service domains. Then we presents several middlewares that are suitable for CoT based platforms and lastly a list of current challenges and issues in design of CoT based middlewares is discussed.Comment: http://www.sciencedirect.com/science/article/pii/S2352864817301268, Digital Communications and Networks, Elsevier (2017

Middleware Technologies for Cloud of Things - a survey

The next wave of communication and applications rely on the new services provided by Internet of Things which is becoming an important aspect in human and machines future. The IoT services are a key solution for providing smart environments in homes, buildings and cities. In the era of a massive number of connected things and objects with a high grow rate, several challenges have been raised such as management, aggregation and storage for big produced data. In order to tackle some of these issues, cloud computing emerged to IoT as Cloud of Things (CoT) which provides virtually unlimited cloud services to enhance the large scale IoT platforms. There are several factors to be considered in design and implementation of a CoT platform. One of the most important and challenging problems is the heterogeneity of different objects. This problem can be addressed by deploying suitable "Middleware". Middleware sits between things and applications that make a reliable platform for communication among things with different interfaces, operating systems, and architectures. The main aim of this paper is to study the middleware technologies for CoT. Toward this end, we first present the main features and characteristics of middlewares. Next we study different architecture styles and service domains. Then we presents several middlewares that are suitable for CoT based platforms and lastly a list of current challenges and issues in design of CoT based middlewares is discussed.Comment: http://www.sciencedirect.com/science/article/pii/S2352864817301268, Digital Communications and Networks, Elsevier (2017

FamTV : an architecture for presence-aware personalized television

Since the advent of the digital era, the traditional TV scenario has rapidly evolved towards an ecosystem comprised of a myriad of services, applications, channels, and contents. As a direct consequence, the amount of available information and configuration options targeted at today's end consumers have become unmanageable. Thus, personalization and usability emerge as indispensable elements to improve our content-overloaded digital homes. With these requirements in mind, we present a way to combine content adaptation paradigms together with presence detection in order to allow a seamless and personalized entertainment experience when watching TV.This work has been partially supported by the Community of Madrid (CAM), Spain under the contract number S2009/TIC-1650.Publicad

Design, analysis and implementation of advanced methodologies to measure the socio-economic impact of personal data in large online services

El ecosistema web es enorme y, en general, se sustenta principalmente en un atributo intangible que sostiene la mayoría de los servicios gratuitos: la explotación de la información personal del usuario. A lo largo de los años, la preocupación por la forma en que los servicios utilizan los datos personales ha aumentado y atraído la atención de los medios de comunicación, gobiernos, reguladores y también de los usuarios. Esta recogida de información personal es hoy en día la principal fuente de ingresos en Internet. Además, por si fuera poco, la publicidad online es la pieza que lo sustenta todo. Sin la existencia de datos personales en comunión con la publicidad online, Internet probablemente no sería el gigante que hoy conocemos. La publicidad online es un ecosistema muy complejo en el que participan múltiples actores. Es el motor principal que genera ingresos en la red, y en pocos años ha evolucionado hasta llegar a miles de millones de usuarios en todo el mundo. Mientras navegan, los usuarios generan datos muy valiosos sobre sí mismos que los anunciantes utilizan después para ofrecerles productos relevantes en los que podrían estar interesados. Se trata de un enfoque bidireccional, ya que los anunciantes pagan a intermediarios para que muestren anuncios al público que, en principio, está más interesado. Sin embargo, este comercio, intercambio y tratamiento de datos personales, además de abrir nuevas vías de publicidad, exponen la privacidad de los usuarios. Esta incesante recopilación y comercialización de la información personal suele quedar tras un muro opaco, donde el usuario generalmente desconoce para qué se utilizan sus datos. Las iniciativas de privacidad y transparencia se han incrementado a lo largo de los años para empoderar al usuario en este negocio que mueve miles de millones de dólares en ingresos. No en vano, tras varios escándalos, como el de Facebook Cambridge Analytica, las empresas y los reguladores se han unido para crear transparencia y proteger a los usuarios de las malas prácticas derivadas del uso de su información personal. Por ejemplo, el Reglamento General de Protección de Datos, es el ejemplo más prometedor de regulación, que afecta a todos los estados miembros de la Unión Europea, abogando por la protección de los usuarios. El contenido de esta tesis tomará como referencia esta legislación. Por todo ello, el propósito de esta tesis consiste en aportar herramientas y metodologías que pongan de manifiesto usos inapropiados de datos personales por las grandes compañías del ecosistema publicitario online, y cree transparencia entre los usuarios, proporcionando, a su vez, soluciones para que se protejan. Así pues, el contenido de esta tesis ofrece diseño, análisis e implementación de metodologías que miden el impacto social y económico de la información personal online en los servicios extensivos de Internet. Principalmente, se centra en Facebook, una de las mayores redes sociales y servicios en la web, que cuenta con más de 2,8B de usuarios en todo el mundo y generó unos ingresos solo en publicidad online de más de 84 mil millones de dólares en el año 2020. En primer lugar, esta tesis presenta una solución, en forma de extensión del navegador llamada FDVT (Data Valuation Tool for Facebook users), para proporcionar a los usuarios una estimación personalizada y en tiempo real del dinero que están generando para Facebook. Analizando el número de anuncios e interacciones en una sesión, el usuario obtiene información sobre su valor dentro de esta red social. La extensión del navegador ha tenido una importante repercusión y adopción tanto por parte de los usuarios, instalándose más de 10k veces desde su lanzamiento público en octubre de 2016, como de los medios de comunicación, apareciendo en más de 100 medios. En segundo lugar, el estudio e investigación de los posibles riesgos asociados al tratamiento de los datos de los usuarios debe seguir también a la creación de este tipo de soluciones. En este contexto, esta tesis descubre y desvela resultados impactantes sobre el uso de la información personal: (i) cuantifica el número de usuarios afectados por el uso de atributos sensibles utilizados para la publicidad en Facebook, utilizando como referencia la definición de datos sensibles del Reglamento General de Protección de Datos. Esta tesis se basa en el uso de Procesamiento de Lenguaje Natural para identificar los atributos sensibles, y posteriormente utiliza el la plataforma de creación de anuncios de Facebook para recuperar el número de usuarios asignados con esta información sensible. Dos tercios de los usuarios de Facebook se ven afectados por el uso de datos personales sensibles que se les atribuyen. Además, la legislación parece no tener efecto en este uso de atributos sensibles por parte de Facebook, y presenta graves riesgos para los usuarios. (ii) Se modela cuál es el número de atributos que no identifican a priori personalmente al usuario y que aun así son suficientes para identificar de forma única a un individuo sobre una base de datos de miles de millones de usuarios, y se demuestra que llegar a un solo usuario es plausible incluso sin conocer datos que lo identifiquen personalmente de ellos mismos. Los resultados demuestran que 22 intereses al azar de un usuario son suficientes para identificarlo unívocamente con un 90% de probabilidad, y 4 si tomamos los menos populares. Por último, esta tesis se ha visto afectada por el estallido de la pandemia del COVID- 19, lo que ha contribuido al análisis de la evolución del mercado de la publicidad en línea con este periodo. La investigación demuestra que el mercado de la publicidad muestra una inelasticidad casi perfecta en la oferta y que cambió su composición debido a un cambio en el comportamiento en línea de los usuarios. También ilustra el potencial que tiene la utilización de los datos de los grandes servicios en línea, dado que ya tienen una alta tasa de adopción, y presenta un protocolo para la localización de contactos que han estado potencialmente expuestos a personas que direon positivo en COVID-19, en contraste con el fracaso de las nuevas aplicaciones de localización de contactos. En conclusión, la investigación de esta tesis muestra el impacto social y económico de la publicidad online y de los grandes servicios online en los usuarios. La metodología utilizada y desplegada sirve para poner de manifiesto y cuantificar los riesgos derivados de los datos personales en los servicios en línea. Presenta la necesidad de tales herramientas y metodologías en consonancia con la nueva legislación y los deseos de los usuarios. Siguiendo estas peticiones, en la búsqueda de transparencia y privacidad, esta tesis muestra soluciones y medidas fácilmente implementables para prevenir estos riesgos y capacitar al usuario para controlar su información personal.The web ecosystem is enormous, and overall it is sustained by an intangible attribute that mainly supports the majority of free services: the exploitation of personal information. Over the years, concerns on how services use personal data have increased and attracted the attention of media and users. This collection of personal information is the primary source of revenue on the Internet nowadays. Furthermore, on top of this, online advertising is the piece that supports it all. Without the existence of personal data in communion with online advertising, the Internet would probably not be the giant we know today. Online advertising is a very complex ecosystem in which multiple stakeholders take part. It is the motor that generates revenue on the web, and it has evolved in a few years to reach billions of users worldwide. While browsing, users generate valuable data about themselves that advertisers later use to offer them relevant products in which users could be interested. It is a two-way approach since advertisers pay intermediates to show ads to the public that is, in principle, most interested. However, this trading, sharing, and processing of personal data and behavior patterns, apart from opening up new advertising ways, expose users’ privacy. This incessant collection and commercialization of personal information usually fall behind an opaque wall, where the user often does not know what their data is used for. Privacy and transparency initiatives have increased over the years to empower the user in this business that moves billions of US dollars in revenue. Not surprisingly, after several scandals, such as the Facebook Cambridge Analytica scandal, businesses and regulators have joined forces to create transparency and protect users against the harmful practices derived from the use of their personal information. For instance, the General Data Protection Regulation (GDPR), is the most promising example of a data protection regulation, affecting all the member states of the European Union (EU), advocating for protecting users. The content of this thesis will use this legislation as a reference. For all these reasons, the purpose of this thesis is to provide tools and methodologies that reveal inappropriate uses of personal data by large companies in the online advertising ecosystem and create transparency among users, providing solutions to protect themselves. Thus, the content of this thesis offers design, analysis, and implementation of methodologies that measure online personal information’s social and economic impact on extensive Internet services. Mainly, it focuses on Facebook (FB), one of the largest social networks and services on the web, accounting with more than 2.8B Monthly Active Users (MAU) worldwide and generating only in online advertising revenue, more than $84B in 2020. First, this thesis presents a solution, in the form of a browser extension called Data Valuation Tool for Facebook users (FDVT), to provide users with a personalized, real-time estimation of the money they are generating for FB. By analyzing the number of ads and interactions in a session, the user gets information on their value within this social network. The add-on has had significant impact and adoption both by users, being installed more than 10k times since its public launch in October 2016, and media, appearing in more than 100 media outlets. Second, the study and research of the potential risks associated with processing users’ data should also follow the creation of these kinds of solutions. In this context, this thesis discovers and unveils striking results on the usage of personal information: (i) it quantifies the number of users affected by the usage of sensitive attributes used for advertising on FB, using as reference the definition of sensitive data from the GDPR. This thesis relies on the use of Natural Language Processing (NLP) to identify sensitive attributes, and it later uses the FB Ads Manager to retrieve the number of users assigned with this sensitive information. Two-thirds of FB users are affected by the use of sensitive personal data attributed to them. Moreover, the legislation seems not to affect this use of sensitive attributes from FB, and it presents severe risks to users. (ii) It models the number of non-Personal Identifiable Information (PII) attributes that are enough to uniquely identify an individual over a database of billions of users and proofs that reaching a single user is plausible even without knowing PII data of themselves. The results demonstrate that 22 interests at random from a user are enough to identify them uniquely with a 90% of probability, and 4 when taking the least popular ones. Finally, this thesis was affected by the outbreak of the COVID-19 pandemic what led to side contribute to the analysis of how the online advertising market evolved during this period. The research shows that the online advertising market shows an almost perfect inelasticity on supply and that it changed its composition due to a change in users’ online behavior. It also illustrates the potential of using data from large online services which already have a high adoption rate and presents a protocol for contact tracing individuals who have been potentially exposed to people who tested positive in COVID-19, in contrast to the failure of newly deployed contact tracing apps. In conclusion, the research for this thesis showcases the social and economic impact of online advertising and extensive online services on users. The methodology used and deployed is used to highlight and quantify the risks derived from personal data in online services. It presents the necessity of such tools and methodologies in line with new legislation and users’ desires. Following these requests, in the search for transparency and privacy, this thesis displays easy implementable solutions and measurements to prevent these risks and empower the user to control their personal information.This work was supported by the Ministerio de Educación, Cultura y Deporte, Spain, through the FPU Grant FPU16/05852, the Ministerio de Ciencia e Innovación, Spain, through the project ACHILLES Grant PID2019-104207RB-I00, the H2020 EU-Funded SMOOTH project under Grant 786741, and the H2020 EU-Funded PIMCITY project under Grant 871370.Programa de Doctorado en Ingeniería Telemática por la Universidad Carlos III de MadridPresidente: David Larrabeiti López.- Secretario: Gregorio Ignacio López López.- Vocal: Noel Cresp