Cyber–Physical–Social Frameworks for Urban Big Data Systems: A Survey

The integration of things’ data on the Web and Web linking for things’ description and discovery is leading the way towards smart Cyber–Physical Systems (CPS). The data generated in CPS represents observations gathered by sensor devices about the ambient environment that can be manipulated by computational processes of the cyber world. Alongside this, the growing use of social networks offers near real-time citizen sensing capabilities as a complementary information source. The resulting Cyber–Physical–Social System (CPSS) can help to understand the real world and provide proactive services to users. The nature of CPSS data brings new requirements and challenges to different stages of data manipulation, including identification of data sources, processing and fusion of different types and scales of data. To gain an understanding of the existing methods and techniques which can be useful for a data-oriented CPSS implementation, this paper presents a survey of the existing research and commercial solutions. We define a conceptual framework for a data-oriented CPSS and detail the various solutions for building human–machine intelligence

City Data Fusion: Sensor Data Fusion in the Internet of Things

Internet of Things (IoT) has gained substantial attention recently and play a significant role in smart city application deployments. A number of such smart city applications depend on sensor fusion capabilities in the cloud from diverse data sources. We introduce the concept of IoT and present in detail ten different parameters that govern our sensor data fusion evaluation framework. We then evaluate the current state-of-the art in sensor data fusion against our sensor data fusion framework. Our main goal is to examine and survey different sensor data fusion research efforts based on our evaluation framework. The major open research issues related to sensor data fusion are also presented.Comment: Accepted to be published in International Journal of Distributed Systems and Technologies (IJDST), 201

A Big-Data based and process-oriented decision support system for traffic management

Data analysis and monitoring of road networks in terms of reliability and performance are valuable but hard to achieve, especially when the analytical information has to be available to decision makers on time. The gathering and analysis of the observable facts can be used to infer knowledge about traffic congestion over time and gain insights into the roads safety. However, the continuous monitoring of live traffic information produces a vast amount of data that makes it difficult for business intelligence (BI) tools to generate metrics and key performance indicators (KPI) in nearly real-time. In order to overcome these limitations, we propose the application of a big-data based and process-centric approach that integrates with operational traffic information systems to give insights into the road network's efficiency. This paper demonstrates how the adoption of an existent process-oriented DSS solution with big-data support can be leveraged to monitor and analyse live traffic data on an acceptable response time basis.publishedVersio

Big Data em cidades inteligentes: um mapeamento sistemático

O conceito de Cidades Inteligentes ganhou maior atenção nos círculos acadêmicos, industriais e governamentais. À medida que a cidade se desenvolve ao longo do tempo, componentes e subsistemas como redes inteligentes, gerenciamento inteligente de água, tráfego inteligente e sistemas de transporte, sistemas de gerenciamento de resíduos inteligentes, sistemas de segurança inteligentes ou governança eletrônica são adicionados. Esses componentes ingerem e geram uma grande quantidade de dados estruturados, semiestruturados ou não estruturados que podem ser processados usando uma variedade de algoritmos em lotes, microlotes ou em tempo real, visando a melhoria de qualidade de vida dos cidadãos. Esta pesquisa secundária tem como objetivo facilitar a identificação de lacunas neste campo, bem como alinhar o trabalho dos pesquisadores com outros para desenvolver temas de pesquisa mais fortes. Neste estudo, é utilizada a metodologia de pesquisa formal de mapeamento sistemático para fornecer uma revisão abrangente das tecnologias de Big Data na implantação de cidades inteligentes

Towards Data Sharing across Decentralized and Federated IoT Data Analytics Platforms

In the past decade the Internet-of-Things concept has overwhelmingly entered all of the fields where data are produced and processed, thus, resulting in a plethora of IoT platforms, typically cloud-based, that centralize data and services management. In this scenario, the development of IoT services in domains such as smart cities, smart industry, e-health, automotive, are possible only for the owner of the IoT deployments or for ad-hoc business one-to-one collaboration agreements. The realization of "smarter" IoT services or even services that are not viable today envisions a complete data sharing with the usage of multiple data sources from multiple parties and the interconnection with other IoT services. In this context, this work studies several aspects of data sharing focusing on Internet-of-Things. We work towards the hyperconnection of IoT services to analyze data that goes beyond the boundaries of a single IoT system. This thesis presents a data analytics platform that: i) treats data analytics processes as services and decouples their management from the data analytics development; ii) decentralizes the data management and the execution of data analytics services between fog, edge and cloud; iii) federates peers of data analytics platforms managed by multiple parties allowing the design to scale into federation of federations; iv) encompasses intelligent handling of security and data usage control across the federation of decentralized platforms instances to reduce data and service management complexity. The proposed solution is experimentally evaluated in terms of performances and validated against use cases. Further, this work adopts and extends available standards and open sources, after an analysis of their capabilities, fostering an easier acceptance of the proposed framework. We also report efforts to initiate an IoT services ecosystem among 27 cities in Europe and Korea based on a novel methodology. We believe that this thesis open a viable path towards a hyperconnection of IoT data and services, minimizing the human effort to manage it, but leaving the full control of the data and service management to the users' will

Promocijas darbs

Elektroniskā versija nesatur pielikumusPromocijas darba mērķis ir sniegt skaidru priekšstatu par bezvadu sensoru tīklu testgultnes platformu izmantošanu un funkcionalitāti, analizēt, kādi testgultnes platformu uzlabojumi ir nepieciešami lai atbalstītu bezvadu sensoru tīklu pētniecību un izstrādi līdz 7. tehnoloģiju gatavības līmenim, kā arī izstrādāt un novērtēt identificētos uzlabojumus. Uzlabojumi ir izstrādāti EDI TestBed testgultnes platformai un novērtēti pamatojoties uz pieciem pabeigtiem un diviem vēl notiekošiem reāliem izmantošanas gadījumiem, diapazonā no 3. līdz 7. tehnoloģiju gatavības līmenim. Katram uzlabojumam tiek sniegtas vadlīnijas un prasības, lai to varētu iekļaut jebkurā saderīgā testgultnes platformā. Promocijas darbā tiek definēts termins "testgultnes platforma" un pamatota šī termina nepieciešamība. Atslēgvārdi: BST; IoT; testgultne; testgultnes platforma; pārskats; sistemātiskais pārskats; bezvadu sensoru tīkli; lietu internetsThe aim of this thesis is to provide a clear view of the usage and functionality of testbed facilities for wireless sensor networks, to analyze what improvements to testbed facilities are needed to support research and development of wireless sensor networks up to Technology Readiness Level 7 and to develop and evaluate the identified improvements. The improvements have been developed for the EDI TestBed facility and evaluated on the basis of five completed and two ongoing real use cases, ranging from Technology Readiness Level 3 to Technology Readiness Level 7. Guidelines and requirements are provided for each improvement so that it can be incorporated into any compatible testbed facility. The thesis defines the term "testbed facility" and justifies the need for this term. Keywords: WSN; IoT; testbed; testbed facility; review; systematic review; wireless sensor networks; internet of things

Analytics-as-a-Service no contexto de plataformas de Big Data para Smart Cities

Dissertação de mestrado integrado em Engenharia e Gestão de Sistemas de InformaçãoO fluxo migratório que se assiste nas últimas décadas, nomeadamente para os centros urbanos, tem como consequência problemas sérios de sustentabilidade, tanto ao nível de recursos naturais, como ao nível da qualidade de vida dos seus habitantes. Sabe-se que 75% da população da União Europeia vive em cidades, e calcula-se que até 2020 este número suba para os 80%, o que preocupa não só as autoridades centrais, como locais. Por outro lado, verifica-se uma mudança de mentalidade, em que o cidadão demonstra interesse em colaborar com as autoridades, para e desta forma se iniciar uma governação participativa. Os desafios que são colocados a este tipo de gestão tornam a mesma impossível por meios tradicionais. No entanto, a proliferação das novas tecnologias já demonstrou a sua mais-valia nas mais variadas áreas, desde que estas sejam bem harmonizadas. Este projeto de dissertação tem como objetivo providenciar a arquitetura bem como a plataforma necessária para colmatar esta necessidade analítica através de um serviço disponibilizado no paradigma as-a-Service. Foi realizado um enquadramento conceptual, sobre a literatura relevante, dando ênfase aos conceitos de Big Data, plataformas, exemplos e arquiteturas de Big Data, tanto ao nível de tratamento como de análise de vastos volumes de dados, dando uma visão do estado de arte em relação à temática em estudo. No que diz respeito ao enquadramento tecnológico foi apresentada a arquitetura BASIS (Arquitetura de Big Data para Smart Cities), que constitui o ponto de partida para esta dissertação, dando continuidade ao trabalho já realizado e seguindo a necessidade identificada pelo autor de aprofundar a camada analítica da arquitetura. Neste sentido foram realizadas análises às plataformas Pentaho, BIRT, SpagoBI e Jaspersoft, de uma forma detalhada, por forma a serem percecionadas as principais características de cada uma delas com vista a identificar a que melhor se enquadra e que responda aos requisitos da arquitetura BASIS. Após a realização do estado da arte foi estabelecida uma arquitetura tecnológica que permitiu a execução de pequenos testes à plataforma SpagoBI onde se constatou que esta era capaz de suprir parte das necessidades analíticas. Seguidamente é feita a proposta de detalhe analítico para a arquitetura BASIS, dando particular atenção à camada conceptual e tecnológica da proposta, colmatando uma das principais lacunas encontradas na literatura, a falta de detalhe tecnológico. Por fim, a proposta de arquitetura foi validada através da integração de funcionalidades SpagoBI no protótipo “SusCity” dando exemplos de utilização do serviço.The migration we are witnessing in recent decades, particularly for urban areas, results in serious problems of sustainability, both in terms of natural resources, as in the quality of life of its habitants. It is known that 75% of the EU population lives in cities, and it is estimated that by 2020 this number will rise to 80%, which concerns not only the central authorities but also, the local ones. On the other hand, it´s possible to see a change of mentality, in which citizens show interest in collaborating with the authorities, and thus to start a participatory governance. The challenges are posed to this type of administration, it impossible to make by traditional ways. However, the proliferation of new technologies has proved its added value in various areas, since they are well harmonized. This dissertation project aims to provide the architecture as well as the necessary platform to fill these analytical needs through a service available in the paradigm as-a- Service. In this work, a literature review on the relevant literature was developed, giving emphasis to the concepts of Big Data, platforms, examples and architectures of Big Data, both in terms of treatment and analysis of vast amounts of data, giving important insights on the state-of-the-art in relation to the topic under study. With regard to the technological environment, it is showed the BASIS architecture designed to support Big Data in a Smart Cities context, which is a starting point for this thesis, continuing the work already done, and following the need identified by the author to go into detail on the analytical layer of the architecture. After this, a detailed analysis was carried out to several platforms like Pentaho, BIRT, SpagoBI and Jaspersoft, in order to see their main features and identify the one that best fits and that responds to the BASIS architecture requirements. After the analysis of the state-of-the-art, it was established a technological architecture that allowed the execution of brief tests to the SpagoBI platform where it was found that it was able to make available many of the needed analytical tasks. The proposed analytical detail specified in the BASIS architecture is, paying particular attention to the conceptual and technological layer, thus fulfilling a major gap in the literature, the lack of technological detail. Finally, the proposed architecture has been validated through the integration of the SpagoBI features in the SusCity prototype, using some user examples

Creating intelligible metrics road traffic analysis

Dissertação de mestrado em Computer ScienceThe increasing pervasiveness and lower cost of electronic devices equipped with sensors is leading to a greater and cheaper availability of localized information. The advent of the internet has brought phenomena such as crowd-sourced maps and related data. The combination of the availability of mobile information, community built maps, with the added convenience of retrieving information over the internet creates the opportunity to contextualize data in new ways. This work takes that opportunity and attempts to generalize the detection of driving events which are deemed problematic as a function of contextual factors, such as neighbouring buildings, areas, amenities, the weather, and the time of day, week or month. In order to research the problem at hand, the issue is first contextualized properly, providing an overview of important factors, namely Smart Cities, Data Fusion, and Machine Learning. That is followed by a chapter concerning the state of the art, that showcases related projects and how the various facets of road traffic expression are being approached. The focus is then turned to creating a solution. At first this consists in aggregating data so as to create a richer context than would be present otherwise, this includes the retrieval from different services, as well as the composition of a unique view of the same driving situation with new dimensions added to it. And then Models were created using different Machine Learning methods, and a comparison of results according to selected and justified evaluation metrics was made. The compared Methods are Decision Tree, Naive Bayes, and Support Vector Machine. The different types of information were evaluated on their own as potential classifiers and then were evaluated together, leading to the conclusion that the various types combined allow for the creation of better models capable of finding problems with more confidence in such results. According to the tests performed the chosen approach can improve the performance over a baseline approach and point out problematic situations with a precision of over 90%. As expected by not using factors concerning the driver state or acceleration the scope of problems which are detected is limited in domain.A expansão e menor custo de dispositivos eletrónicos equipados com sensores está a levar a uma maior e mais barata disponibilidade de informação localizada. O advento da internet criou fenómenos como a criação de mapas e dados relacionados gerados por comunidades. A combinação da disponibilidade de informação móvel e mapas construídos pela comunidade, em conjunto com uma obtenção de informação através da internet mais conveniente, criou a oportunidade de contextualizar os dados de novas maneiras. Este trabalho faz uso dessa oportunidade e tenta generalizar eventos de condução que são considerados problemáticos em função de factores contextuais, tais como a presença de edifícios, áreas, e comodidades na vizinhança, o clima, e a hora do dia, a semana, ou o mês. De modo a investigar esta questão, o problema é contextualizado como emergente no tópico de Cidades Inteligentes, e explorado com recurso a Fusão de Dados e a Aprendizagem Máquina. O estado da arte é exposto, através de projectos relacionados à expressão do tráfego rodoviário, dando relevo às várias facetas até então investigadas por outros autores de modo a enquadrar o trabalho presente. Dado o enquadramento e concretização do problema, é proposta uma solução. Esta solução passa por inicialmente agregar dados de modo a enriquecer o contexto, incluindo a recolha destes de vários serviços, e uma composição dos dados recolhidos numa perspectiva única referente a uma situação de condução. Após este enriquecimento dos dados, são criados modelos com base em diferentes técnicas de Aprendizagem Máquina. Os métodos utilizados são Decision Tree, Naive Bayes, e Support Vector Machine. Os resultados conseguidos com estes modelos são depois comparados de acordo com as métricas de avaliação seleccionadas. Uma comparação foi feita também com diferentes tipos de informação separadamente e também em conjunto, levando à conclusão de que os vários tipos combinados permitem a criação de melhores modelos capazes de encontrar problemas com mais confiança nos resultados produzidos. De acordo com os testes executados a abordagem escolhida consegue melhorar resultados de um modelo base e descobrir situações problemáticas de condução com uma precisão acima dos 90%. No entanto, como seria de esperar, o âmbito dos problemas detectados tem um domínio limitado aos aspectos seleccionados

Reputation-aware Trajectory-based Data Mining in the Internet of Things (IoT)

Internet of Things (IoT) is a critically important technology for the acquisition of spatiotemporally dense data in diverse applications, ranging from environmental monitoring to surveillance systems. Such data helps us improve our transportation systems, monitor our air quality and the spread of diseases, respond to natural disasters, and a bevy of other applications. However, IoT sensor data is error-prone due to a number of reasons: sensors may be deployed in hazardous environments, may deplete their energy resources, have mechanical faults, or maybe become the targets of malicious attacks by adversaries. While previous research has attempted to improve the quality of the IoT data, they are limited in terms of better realization of the sensing context and resiliency against malicious attackers in real time. For instance, the data fusion techniques, which process the data in batches, cannot be applied to time-critical applications as they take a long time to respond. Furthermore, context-awareness allows us to examine the sensing environment and react to environmental changes. While previous research has considered geographical context, no related contemporary work has studied how a variety of sensor context (e.g., terrain elevation, wind speed, and user movement during sensing) can be used along with spatiotemporal relationships for online data prediction. This dissertation aims at developing online methods for data prediction by fusing spatiotemporal and contextual relationships among the participating resource-constrained mobile IoT devices (e.g. smartphones, smart watches, and fitness tracking devices). To achieve this goal, we first introduce a data prediction mechanism that considers the spatiotemporal and contextual relationship among the sensors. Second, we develop a real-time outlier detection approach stemming from a window-based sub-trajectory clustering method for finding behavioral movement similarity in terms of space, time, direction, and location semantics. We relax the prior assumption of cooperative sensors in the concluding section. Finally, we develop a reputation-aware context-based data fusion mechanism by exploiting inter sensor-category correlations. On one hand, this method is capable of defending against false data injection by differentiating malicious and honest participants based on their reported data in real time. On the other hand, this mechanism yields a lower data prediction error rate