On the Data Freshness for IoT Traffic Modelling in Real-Time Emergency Observation Systems

Internet of things (IoT) and fog computing based observation systems are gaining more importance as Internet becomes the main infrastructure to augment the pervasiveness in remote monitoring of the physical world. Considering the explosion in the number of connected ”things”, the increase of data traffic density on interconnection devices (i.e., IoT gateways) becomes an important problem for scalable real-time emergency detection and monitoring. Thus, data traffic analysis and modeling of fog services become an important research area to get more insights into real-time behavior of such systems. The outcomes of such analysis are important for prediction of IoT system behavior in a given network topology. In this paper, we elaborate on an architectural solution for periodic data acquisition from a wireless sensor network(WSN). To this end, we propose a publish/subscribe (P/S) based observation scheme which simultaneously interconnects clients to different kind of sensor devices over a fog layer service. Then, we examine the data freshness which is a critical traffic modeling parameter for real-time emergency observation. With using such scheme, we devise an analysis for understanding the behavior of the overall system in the context of data freshness. The results obtained from our experimental setup illustrate the appropriateness of freshness time calculation methods for obtaining the required service quality

Manejo de cambios en la calidad de las fuentes en sistemas de integración de datos

Los Sistemas de Integración de Datos (DIS) integran información desde un conjunto de Fuentes de Datos heterogéneas y autónomas, y proveen dicha información a un conjunto de Vistas de Usuario. Consideramos un sistema donde se toman en cuenta las propiedades de calidad. En las fuentes existen los valores reales de las propiedades de calidad y en el sistema integrado existen los valores requeridos de estas propiedades. En este tipo de sistema, considerando la gran cantidad posible de fuentes y su autonomía, aparece un nuevo problema: los cambios en la calidad de las fuentes. Los valores reales de los elementos de las fuentes pueden cambiar con mucha frecuencia y de forma impredecible. Nos interesan las consecuencias que pueden tener los cambios en la calidad de las fuentes sobre la calidad global del sistema, e incluso sobre el esquema del DIS y la forma de procesar su información. Analizamos estas consecuencias basándonos en las diferentes posibilidades existentes para manejar los cambios en los esquemas de las fuentes en sistemas de este tipo. Además estudiamos dos propiedades en particular; frescura y precisión, y definimos estrategias para el manejo de los cambios en estas propiedades

Optimizing the Replication of Multi-Quality Web Applications Using ACO and WoLF

This thesis presents the adaptation of Ant Colony Optimization to a new NP-hard problem involving the replication of multi-quality database-driven web applications (DAs) by a large application service provider (ASP). The ASP must assign DA replicas to its network of heterogeneous servers so that user demand is satisfied and replica update loads are minimized. The algorithm proposed, AntDA, for solving this problem is novel in several respects: ants traverse a bipartite graph in both directions as they construct solutions, pheromone is used for traversing from one side of the bipartite graph to the other and back again, heuristic edge values change as ants construct solutions, and ants may sometimes produce infeasible solutions. Experiments show that AntDA outperforms several other solution methods, but there was room for improvement in the convergence rates of the ants. Therefore, in an attempt to achieve the goals of faster convergence and better solution values for larger problems, AntDA was combined with the variable-step policy hill-climbing algorithm called Win or Learn Fast (WoLF). In experimentation, the addition of this learning algorithm in AntDA provided for faster convergence while outperforming other solution methods

Data freshness and data accuracy :a state of the art

In a context of Data Integration Systems (DIS) providing access to large amounts of data extracted and integrated from autonomous data sources, users are highly concerned about data quality. Traditionally, data quality is characterized via multiple quality factors. Among the quality dimensions that have been proposed in the literature, this report analyzes two main ones: data freshness and data accuracy. Concretely, we analyze the various definitions of both quality dimensions, their underlying metrics and the features of DIS that impact their evaluation. We present a taxonomy of existing works proposed for dealing with both quality dimensions in several kinds of DIS and we discuss open research problems

Data quality maintenance in Data Integration Systems

A Data Integration System (DIS) is an information system that integrates data from a set of heterogeneous and autonomous information sources and provides it to users. Quality in these systems consists of various factors that are measured in data. Some of the usually considered ones are completeness, accuracy, accessibility, freshness, availability. In a DIS, quality factors are associated to the sources, to the extracted and transformed information, and to the information provided by the DIS to the user. At the same time, the user has the possibility of posing quality requirements associated to his data requirements. DIS Quality is considered as better, the nearer it is to the user quality requirements. DIS quality depends on data sources quality, on data transformations and on quality required by users. Therefore, DIS quality is a property that varies in function of the variations of these three other properties. The general goal of this thesis is to provide mechanisms for maintaining DIS quality at a level that satisfies the user quality requirements, minimizing the modifications to the system that are generated by quality changes. The proposal of this thesis allows constructing and maintaining a DIS that is tolerant to quality changes. This means that the DIS is constructed taking into account previsions of quality behavior, such that if changes occur according to these previsions the system is not affected at all by them. These previsions are provided by models of quality behavior of DIS data, which must be maintained up to date. With this strategy, the DIS is affected only when quality behavior models change, instead of being affected each time there is a quality variation in the system. The thesis has a probabilistic approach, which allows modeling the behavior of the quality factors at the sources and at the DIS, allows the users to state flexible quality requirements (using probabilities), and provides tools, such as certainty, mathematical expectation, etc., that help to decide which quality changes are relevant to the DIS quality. The probabilistic models are monitored in order to detect source quality changes, strategy that allows detecting changes on quality behavior and not only punctual quality changes. We propose to monitor also other DIS properties that affect its quality, and for each of these changes decide if they affect the behavior of DIS quality, taking into account DIS quality models. Finally, the probabilistic approach is also applied at the moment of determining actions to take in order to improve DIS quality. For the interpretation of DIS situation we propose to use statistics, which include, in particular, the history of the quality models

Medición de la calidad de datos :un enfoque parametrizable

En este momento, las bases de datos constituyen uno de los principales activos de las empresas. Los problemasde calidad de datos inducen a errores o falta de precisión en el análisis de los mismos, lo cual puede derivar en un alto costo para la empresa. En tal sentido, en esta tesis nos enfocamos en el estudio de mecanismos de medición de la calidad de los datos. Presentamos un estado del arte sobre medición de algunas dimensiones de calidad y experimentamos en una aplicación real de un área de negocio financiera, con el dominio de aplicación CRM, en un esquema de replicación de bases de datos. Para medir la calidad ponemos en práctica una metodología en la que las métricas de calidad se obtienen refinando las metas de calidad de la organización. Como resultado obtuvimos una biblioteca de métodos de medición de la calidad y una base de datos con las medidas tomadas para la aplicación financiera. Los métodos propuestos son parametrizables y extensibles, pudiendo ser utilizados en diferentes aplicaciones. Nuestro enfoque puede ser utilizado en las empresas con diferentes objetivos: estadísticas, particionamiento de las tablas de acuerdo a su calidad, mejoras en la explotación de la información, tareas de data-cleaning, entre otros

Using latency-recency profiles for data delivery on the web

1 Introduction Recent technological advances and the rapid growth of the Internet have increased the number of clients who access objects from remote servers on the Web. Remote data access is often characterized by high latency due to network traffic and remote server workloads. Data at remote servers is often characterized by frequent updates. For example, stock quotes may be updated as frequently as once per minute[12]

Une approche flexible et décentralisée du traitement de requêtes dans les systèmes géo-distribués

This thesis studies the design of query processing systems, across a diversity of geo-distributed settings. Optimising performance metrics such as response time, freshness, or operational cost involves design decisions, such as what derived state (e.g., indexes, materialised views, or caches) to maintain, and how to distribute and where to place the corresponding computation and state. These metrics are often in tension, and the trade-offs depend on the specific application and/or environment. This requires the ability to adapt the query engine's topology and architecture, and the placement of its components. This thesis makes the following contributions: - A flexible architecture for geo-distributed query engines, based on components connected in a bidirectional acyclic graph. - A common microservice abstraction and API for these components, the Query Processing Unit (QPU). A QPU encapsulates some primitive query processing task. Multiple QPU types exist, which can be instantiated and composed into complex graphs. - A model for constructing modular query engine architectures as a distributed topology of QPUs, enabling flexible design and trade-offs between performance metrics. - Proteus, a QPU-based framework for constructing and deploying query engines. - Representative deployments of Proteus and experimental evaluation thereof.Cette thèse présente l'étude de la conception de systèmes de traitement de requêtes dans divers cadres géo-distribués. L'optimisation des mesures de performance telles que le temps de réponse, la fraîcheur ou le coût opérationnel implique des décisions de conception tel que le choix de l’état dérivé (indices, vues matérialisées, caches par ex.) à construire et maintenir, et la distribution et le placement de ces derniers et de leurs calculs. Ces métriques sont souvent opposées et les compromis dépendent de l'application et/ou de la spécificité de l'environnement. La capacité d'adapter la topologie et l'architecture du système de traitement de requêtes devient alors essentielle, ainsi que le placement de ses composants. Cette thèse apporte les contributions suivantes : - Une architecture flexible pour les systèmes de traitement de requêtes géo-distribués, basée sur des composants connectés dans un graphe bidirectionnel acyclique. - Une abstraction de micro-service et une API communes pour ces composants, le Query Processing Unit (QPU). Un QPU encapsule une tâche de traitement de requête primitive. Il existe plusieurs types de QPU qui peuvent être instanciés et composés en graphes complexes. - Un modèle pour construire des architectures de systèmes de traitement de requêtes modulaires composées d’une topologie distribuée de QPUs, permettant une conception flexible et des compromis selon les mesures de performance visées. - Proteus, un framework basé sur les QPU, permettant la construction et le déploiement de systèmes de traitement de requêtes. - Déploiements représentatifs de systèmes de traitement de requêtes à l'aide de Proteus, et leur évaluation expérimentale