Big data in SATA Airline: finding new solutions for old problems

With the rapid growth of operational data needed in airlines and the value that can be attributed to knowledge extracted from these data, airlines have already realized the importance of technologies and methodologies associated with the concept of big data. In this article we present the case study of SATA Airlines. The operational and the decision support systems are described as well as the perspectives of using these new technologies to support knowledge creation and aid the solution of problems in this specific company. The proposed system provides a new operational environment.info:eu-repo/semantics/publishedVersio

Noise Data Visualization and Identification Project

This project aims to produce a space and time map of noise levels within a city using data gathered from sensors, with the goal of identifying noise hot spots and quiet zones. It also includes a noise identification module that attempts to classify reported sound data

Noise Data Visualization and Identification Project

This project aims to produce a space and time map of noise levels within a city using data gathered from sensors, with the goal of identifying noise hot spots and quiet zones. It also includes a noise identification module that attempts to classify reported sound data

Survey on graph databases and their applications

The idea of representing information or logic through graphs extends back to 18th century by Leonard Euler. The graph theory is one of the fundamentals of mathematics but it hasn’t been until the recent years where the graph-based data models started to get applied to various aspects of the information world. The relational database models have dominated the database industry for the last fifty years. The precedence of relational models can be attributed to their storage space efficiency, reliability and the created abstraction between the database and the user. Even though relational databases have succeeded for a long time, with the rise of web, big data and unstructured data, the need for new data storage models became apparent. Graph based database models solve most of the shortcomings of relational models and they are quickly gaining popularity among various industries. In this report I will analyze the history of database models starting from the relational models and compare these models to the newer NoSQL storage models focusing on graph-based storage. I will give examples of successful implementation of such storage models and finally I will talk about unique applications of these graph databasesInformatio

Back-end reference architecture for smart water meter data gathering service

Abstract. The Finnish waterworks industry is on the brink of digitalization. Currently, many of them have started to convert their water meters to smart water meters. However, there is yet no suitable solution for gathering the IoT data from these smart water meters. To answer their arising needs, many pilots and workshops have been conducted. Those pilots have yielded some basic ground rules for their use cases. In this study, those ground rules have been gathered as a set of requirement categories. The categories are studied and analyzed in order to establish a reference architecture for IoT data-gathering systems suitable for waterworks. Using the requirements and the reference architecture, an information system, Dataservice, was implemented by Vesitieto Oy. The system gathers the IoT data and visualizes it to waterworks’ employees. The System was deployed in Microsoft’s cloud service, but other cloud vendors were examined as well. The system has a two-folded database system, the data required by the system, like users and user groups, are held in the SQL database. The IoT-data is held in a NoSQL database. The selected NoSQL database provider was MongoDB as it could be integrated with the cloud provider.Etäluettavien vesimittareiden datapalvelun viitearkkitehtuuri. Tiivistelmä. Suomen vesihuolto on digitalisaation partaalla. Tällä hetkellä monet vesilaitokset ovat alkaneet vaihtaa vanhoja analogisia vesimittareitaan älykkäiksi vesimittareiksi. Vesihuoltolaitokset eivät kuitenkaan ole löytäneet kaikille sopivaa ratkaisua IoT-tiedon keräämiseen älykkäistä vesimittareista. Vastatakseen vesilaitosten tarpeisiin, monia pilotteja ja työpajoja on järjestetty eri yhteistyökumppaneiden kanssa. Näistä eri piloteista on muodostunut käsitys siitä, kuinka vesimittareiden digitalisaatio voidaan ratkaista vesilaitoksilla. Tässä tutkimuksessa eri laitosten väliset perussäännöt on koottu ohjelmistovaatimusluokiksi. Näitä luokkia tutkitaan ja analysoidaan vesilaitoksille sopivan IoT-tiedonkeruujärjestelmän viitearkkitehtuurin luomiseksi. Vaatimuksia ja viitearkkitehtuuria hyödyntäen Vesitieto Oy toteutti tietojärjestelmän nimeltään ”Dataservice”. Järjestelmä kerää IoT-tiedot ja visualisoi ne vesilaitosten työntekijöille. Järjestelmä otettiin käyttöön Microsoftin pilvipalvelussa, mutta myös muita pilvipalvelun palveluntarjoajia tutkittiin. Järjestelmässä on kaksiportainen tietokantajärjestelmä. Järjestelmän tarvitsemat tiedot kuten käyttäjät sekä käyttäjäryhmät pidetään SQLtietokannassa ja IoT-tiedot pidetään NoSQL-tietokannassa. Valittu NoSQL tietokantajärjestelmä oli MongoDB, koska se voitiin integroida pilvipalveluntarjoajan kanssa

A Data-driven Methodology Towards Mobility- and Traffic-related Big Spatiotemporal Data Frameworks

Human population is increasing at unprecedented rates, particularly in urban areas. This increase, along with the rise of a more economically empowered middle class, brings new and complex challenges to the mobility of people within urban areas. To tackle such challenges, transportation and mobility authorities and operators are trying to adopt innovative Big Data-driven Mobility- and Traffic-related solutions. Such solutions will help decision-making processes that aim to ease the load on an already overloaded transport infrastructure. The information collected from day-to-day mobility and traffic can help to mitigate some of such mobility challenges in urban areas. Road infrastructure and traffic management operators (RITMOs) face several limitations to effectively extract value from the exponentially growing volumes of mobility- and traffic-related Big Spatiotemporal Data (MobiTrafficBD) that are being acquired and gathered. Research about the topics of Big Data, Spatiotemporal Data and specially MobiTrafficBD is scattered, and existing literature does not offer a concrete, common methodological approach to setup, configure, deploy and use a complete Big Data-based framework to manage the lifecycle of mobility-related spatiotemporal data, mainly focused on geo-referenced time series (GRTS) and spatiotemporal events (ST Events), extract value from it and support decision-making processes of RITMOs. This doctoral thesis proposes a data-driven, prescriptive methodological approach towards the design, development and deployment of MobiTrafficBD Frameworks focused on GRTS and ST Events. Besides a thorough literature review on Spatiotemporal Data, Big Data and the merging of these two fields through MobiTraffiBD, the methodological approach comprises a set of general characteristics, technical requirements, logical components, data flows and technological infrastructure models, as well as guidelines and best practices that aim to guide researchers, practitioners and stakeholders, such as RITMOs, throughout the design, development and deployment phases of any MobiTrafficBD Framework. This work is intended to be a supporting methodological guide, based on widely used Reference Architectures and guidelines for Big Data, but enriched with inherent characteristics and concerns brought about by Big Spatiotemporal Data, such as in the case of GRTS and ST Events. The proposed methodology was evaluated and demonstrated in various real-world use cases that deployed MobiTrafficBD-based Data Management, Processing, Analytics and Visualisation methods, tools and technologies, under the umbrella of several research projects funded by the European Commission and the Portuguese Government.A população humana cresce a um ritmo sem precedentes, particularmente nas áreas urbanas. Este aumento, aliado ao robustecimento de uma classe média com maior poder económico, introduzem novos e complexos desafios na mobilidade de pessoas em áreas urbanas. Para abordar estes desafios, autoridades e operadores de transportes e mobilidade estão a adotar soluções inovadoras no domínio dos sistemas de Dados em Larga Escala nos domínios da Mobilidade e Tráfego. Estas soluções irão apoiar os processos de decisão com o intuito de libertar uma infraestrutura de estradas e transportes já sobrecarregada. A informação colecionada da mobilidade diária e da utilização da infraestrutura de estradas pode ajudar na mitigação de alguns dos desafios da mobilidade urbana. Os operadores de gestão de trânsito e de infraestruturas de estradas (em inglês, road infrastructure and traffic management operators — RITMOs) estão limitados no que toca a extrair valor de um sempre crescente volume de Dados Espaciotemporais em Larga Escala no domínio da Mobilidade e Tráfego (em inglês, Mobility- and Traffic-related Big Spatiotemporal Data —MobiTrafficBD) que estão a ser colecionados e recolhidos. Os trabalhos de investigação sobre os tópicos de Big Data, Dados Espaciotemporais e, especialmente, de MobiTrafficBD, estão dispersos, e a literatura existente não oferece uma metodologia comum e concreta para preparar, configurar, implementar e usar uma plataforma (framework) baseada em tecnologias Big Data para gerir o ciclo de vida de dados espaciotemporais em larga escala, com ênfase nas série temporais georreferenciadas (em inglês, geo-referenced time series — GRTS) e eventos espacio- temporais (em inglês, spatiotemporal events — ST Events), extrair valor destes dados e apoiar os RITMOs nos seus processos de decisão. Esta dissertação doutoral propõe uma metodologia prescritiva orientada a dados, para o design, desenvolvimento e implementação de plataformas de MobiTrafficBD, focadas em GRTS e ST Events. Além de uma revisão de literatura completa nas áreas de Dados Espaciotemporais, Big Data e na junção destas áreas através do conceito de MobiTrafficBD, a metodologia proposta contem um conjunto de características gerais, requisitos técnicos, componentes lógicos, fluxos de dados e modelos de infraestrutura tecnológica, bem como diretrizes e boas práticas para investigadores, profissionais e outras partes interessadas, como RITMOs, com o objetivo de guiá-los pelas fases de design, desenvolvimento e implementação de qualquer pla- taforma MobiTrafficBD. Este trabalho deve ser visto como um guia metodológico de suporte, baseado em Arqui- teturas de Referência e diretrizes amplamente utilizadas, mas enriquecido com as característi- cas e assuntos implícitos relacionados com Dados Espaciotemporais em Larga Escala, como no caso de GRTS e ST Events. A metodologia proposta foi avaliada e demonstrada em vários cenários reais no âmbito de projetos de investigação financiados pela Comissão Europeia e pelo Governo português, nos quais foram implementados métodos, ferramentas e tecnologias nas áreas de Gestão de Dados, Processamento de Dados e Ciência e Visualização de Dados em plataformas MobiTrafficB

Privacy Preserved Model Based Approaches for Generating Open Travel Behavioural Data

Location-aware technologies and smart phones are fast growing in usage and adoption as a medium of service request and delivery of daily activities. However, the increasing usage of these technologies has birthed new challenges that needs to be addressed. Privacy protection and the need for disaggregate mobility data for transportation modelling needs to be balanced for applications and academic research. This dissertation focuses on developing modern privacy mechanisms that seek to satisfy requirements on privacy and data utility for fine-grained travel behavioural modelling applications using large-scale mobility data. To accomplish this, we review the challenges and opportunities that are needed to be solved in order to harness the full potential of “Big Transportation Data”. Also, we perform a quantitative evaluation on the degree of privacy that are provided by popular location anonymization techniques when undertaken on sensitive location data (i.e. homes, offices) of a travel survey. As a step to solve the trade-off between privacy and utility, we develop a differentially-private generative model for simultaneously synthesizing both socio-economic attributes and sequences of activity diary. Adversarial attack models are proposed and tested to evaluate the effectiveness of the proposed system against privacy attacks. The results show that datasets from the developed privacy enhancing system can be used for travel behavioural modelling with satisfactory results while ensuring an acceptable level of privacy