ETL and analysis of IoT data using OpenTSDB, Kafka, and Spark

Master's thesis in Computer scienceThe Internet of Things (IoT) is becoming increasingly prevalent in today's society. Innovations in storage and processing methodologies enable the processing of large amounts of data in a scalable manner, and generation of insights in near real-time. Data from IoT are typically time-series data but they may also have a strong spatial correlation. In addition, many time-series data are deployed in industries that still place the data in inappropriate relational databases. Many open-source time-series databases exist today with inspiring features in terms of storage, analytic representation, and visualization. Finding an efficient method to migrate data into a time-series database is the first objective of the thesis. In recent decades, machine learning has become one of the backbones of data innovation. With the constantly expanding amounts of information available, there is good reason to expect that smart data analysis will become more pervasive as an essential element for innovative progress. Methods for modeling time-series data in machine learning and migrating time-series data from a database to a big data machine learning framework, such as Apache Spark, is explored in this thesis

How functional programming mattered

In 1989 when functional programming was still considered a niche topic, Hughes wrote a visionary paper arguing convincingly ‘why functional programming matters’. More than two decades have passed. Has functional programming really mattered? Our answer is a resounding ‘Yes!’. Functional programming is now at the forefront of a new generation of programming technologies, and enjoying increasing popularity and influence. In this paper, we review the impact of functional programming, focusing on how it has changed the way we may construct programs, the way we may verify programs, and fundamentally the way we may think about programs

VERDICTS: Visual Exploratory Requirements Discovery and Injection for Comprehension and Testing of Software

We introduce a methodology and research tools for visual exploratory software analysis. VERDICTS combines exploratory testing, tracing, visualization, dynamic discovery and injection of requirements specifications into a live quick-feedback cycle, without recompilation or restart of the system under test. This supports discovery and verification of software dynamic behavior, software comprehension, testing, and locating the defect origin. At its core, VERDICTS allows dynamic evolution and testing of hypotheses about requirements and behavior, by using contracts as automated component verifiers. We introduce Semantic Mutation Testing as an approach to evaluate concordance of automated verifiers and the functional specifications they represent with respect to existing implementation. Mutation testing has promise, but also has many known issues. In our tests, both black-box and white-box variants of our Semantic Mutation Testing approach performed better than traditional mutation testing as a measure of quality of automated verifiers

Third international workshop on Authoring of adaptive and adaptable educational hypermedia (A3EH), Amsterdam, 18-22 July, 2005

The A3EH follows a successful series of workshops on Adaptive and Adaptable Educational Hypermedia. This workshop focuses on models, design and authoring of AEH, on assessment of AEH, conversion between AEH and evaluation of AEH. The workshop has paper presentations, poster session and panel discussions

Conversando con la ciudad: Una plataforma de Internet de las Cosas para interacción entre ciudadanos y espacios públicos

Trabajo de Fin de Máster en Máster en Internet de las Cosas, Facultad de Informática UCM, Departamento de Sistemas Informáticos y Computación, Curso 2019/2020This work presents the development of a technological platform for cities whose objective is to allow citizens and tourists to “communicate with it” in an interactive way, through voice commands and without a predetermined script. These characteristics allow users to guide their own learning process in order to obtain information and better understand the history of the cities where they live or visit. More than a strictly technological approach, the project invites the user to explore the different narratives about cities and to positively occupy their public spaces. The platform consists of four technologies that work together: a web application, a mobile application, IoT devices and cognitive assistants that operate with Artificial Intelligence. Different concepts, programming languages and tools were combined to create the project, such as software development frameworks, databases, Bluetooth Low Energy protocol-based devices and cloud computing services. For this first version of the project, the idea is that interested users or institutions can add devices to the platform – and the mechanism for this is presented throughout the chapters of this work.Este trabajo presenta el desarrollo de una plataforma tecnológica para ciudades cuyo objetivo es permitir que ciudadanos y turistas “se comuniquen con ella” de forma interactiva, a través de comandos de voz y sin un guion predeterminado. Estas características permiten a los usuarios orientar su propio proceso de aprendizaje para obtener información y comprender mejor la historia de las ciudades donde viven o que visitan. Más que un enfoque estrictamente tecnológico, el proyecto invita al usuario a explorar las diferentes narrativas sobre las ciudades y a ocupar positivamente sus espacios públicos. La plataforma consta de cuatro tecnologías que funcionan juntas: una aplicación web, una aplicación móvil, dispositivos IoT y asistentes cognitivos que operan con Inteligencia Artificial. Diferentes conceptos, lenguajes de programación y herramientas fueron combinados para crear el proyecto, como frameworks de desarrollo de software, bases de datos, dispositivos basados en el protocolo Bluetooth Low Energy y servicios de computación en la nube. Para esta primera versión del proyecto, la idea es que los usuarios o instituciones interesados puedan agregar dispositivos a la plataforma – y el mecanismo para esto se presenta a lo largo de los capítulos de este trabajoDepto. de Sistemas Informáticos y ComputaciónFac. de InformáticaTRUEunpu

Essays on the use of e-Learning in statistics and the implementation of statistical software

Die vorliegende Doktorarbeit bündelt die Veröffentlichungen des Autors und seiner Koautoren zu den Themen e-Learning und statistischer Software. Die Kapitel 2 bis 5 sind Aspekten des e-Learning gewidmet, die Kapitel 6 bis 9 beschreiben die Entwicklung der statistischen Programmiersprache Yxilon. In Kapitel 2, Koautoren Wolfgang Härdle und Sigbert Klinke, wird erörtert, ob und wie computerbasierte Elemente in den Kanon der methodischen Bildung integriert werden sollen und wo die Grenzen des e-Learning in der Statistik-Ausbildung liegen. Kapitel 3, Koautoren Wolfgang Härdle und Sigbert Klinke, gibt Einschätzungen verschiedener e-Learning Plattformen und beschreibt Punkte, die bei der Entwicklung von e-Learning Plattformen berücksichtigt werden sollten. Kapitel 4, geschrieben mit Wolfgang Härdle und Sigbert Klinke, diskutiert zwei Veröffentlichungen in der "International Statistical Review", die eine technische Lösung für die Verbesserung des Verständnisses der Statistik-Lehre vorstellen. Kapitel 5, Koautoren Wolfgang Härdle und Sigbert Klinke, beschreibt die Anwendung von Web-Techniken für die Lehre in Statistik. Weiterhin stellt es die Quantnet Plattform vor, eine Plattform für die Verwaltung von Programmen und Daten. In Kapitel 6, Koautoren Wolfgang Härdle und Sigbert Klinke, diskutieren die Autoren die Anforderungen an eine Statistical Engine. Kapitel 7, geschrieben mit Yuval Guri und Sigbert Klinke, erläutert die Ideen, die zur Re-Implementierung der XploRe Sprache geführt haben und diskutiert ausgewählte technische Aspekte der Yxilon Plattform wie Objektdatenbank und die Erzeugung von kompilierbarem Code für Hochsprachen. In Kapitel 8, Koautoren Wolfgang Härdle und Sigbert Klinke, wird die implementierte Client-Server Struktur beschrieben. Server und Kommunikationsprotokoll werden zusammen mit dem entwickelten Client und der Grafik-Engine beschrieben. Das letzte Kapitel, beschreibt die Struktur der Yxilon Plattform in ihrer jetzigen Form.The following doctoral thesis collects the papers the author has written with his coauthors on e-Learning and statistical software. The chapters 2 to 5 are devoted to selected aspects of e-Learning, the chapters 6 to 9 describe the development of the statistical programming environment Yxilon. In chapter 2, coauthored by Wolfgang Härdle and Sigbert Klinke, the question whether and how computational elements should be integrated into the canon of methodological education and where e-techniques have their limits in statistics education is discussed. Chapter 3, coauthored by Wolfgang Härdle and Sigbert Klinke, gives reviews of different e-learning platforms for statistics and reveals facts that may be taken into account for future e-learning platforms in statistics and related fields. Chapter 4, written with Wolfgang Härdle and Sigbert Klinke, discusses two papers published in International Statistical Review which both offer a technical solution to improve the understanding of statistics by students. Chapter 5, coauthored by Wolfgang Härdle and Sigbert Klinke, describes web-related techniques for teaching statistics. It furthermore introduces the Quantnet platform, a framework to manage scientific code and data. In chapter 6, coauthored by Wolfgang Härdle and Sigbert Klinke, the requirements for a statistical engine are discussed. Chapter 7, written jointly with Yuval Guri and Sigbert Klinke, explains ideas which led to the reimplementation of the XploRe language. In chapter 8, coauthored by Wolfgang Härdle and Sigbert Klinke, the implemented client/server structure of the Yxilon platform is laid out in terms of technical features. The server and the communication protocol are described together with the developed Java client featuring the Jasplot graphics engine. Finally chapter 9 describes the structure of the Yxilon environment in its present form