Data semantic enrichment for complex event processing over IoT Data Streams

This thesis generalizes techniques for processing IoT data streams, semantically enrich data with contextual information, as well as complex event processing in IoT applications. A case study for ECG anomaly detection and signal classification was conducted to validate the knowledge foundation

Asynchronous Validity Resolution in Sequentially Consistent Shared Virtual Memory

Shared Virtual Memory (SVM) is an effort to provide a mechanism for a distributed system, such as a cluster, to execute shared memory parallel programs. Unfortunately, SVM has performance problems due to its underlying distributed architecture. Recent developments have increased performance of SVM by reducing communication. Unfortunately this performance gain was only possible by increasing programming complexity and by restricting the types of programs allowed to execute in the system. Validity resolution is the process of resolving the validity of a memory object such as a page. Current SVM systems use synchronous or deferred validity resolution techniques in which user processing is blocked during the validity resolution process. This is the case even when resolving validity of false shared variables. False-sharing occurs when two or more processes access unrelated variables stored within the same shared block of memory and at least one of the processes is writing. False sharing unnecessarily reduces overall performance of SVM systems?because user processing is blocked during validity resolution although no actual data dependencies exist. This thesis presents Asynchronous Validity Resolution (AVR), a new approach to SVM which reduces the performance losses associated with false sharing while maintaining the ease of programming found with regular shared memory parallel programming methodology. Asynchronous validity resolution allows concurrent user process execution and data validity resolution. AVR is evaluated by com-paring performance of an application suite using both an AVR sequentially con-sistent SVM system and a traditional sequentially consistent (SC) SVM system. The results show that AVR can increase performance over traditional sequentially consistent SVM for programs which exhibit false sharing. Although AVR outperforms regular SC by as much as 26%, performance of AVR is dependent on the number of false-sharing vs. true-sharing accesses, the number of pages in the program’s working set, the amount of user computation that completes per page request, and the internodal round-trip message time in the system. Overall, the results show that AVR could be an important member of the arsenal of tools available to parallel programmers

Real time stream processing for internet of things

06.03.2018 tarihli ve 30352 sayılı Resmi Gazetede yayımlanan “Yükseköğretim Kanunu İle Bazı Kanun Ve Kanun Hükmünde Kararnamelerde Değişiklik Yapılması Hakkında Kanun” ile 18.06.2018 tarihli “Lisansüstü Tezlerin Elektronik Ortamda Toplanması, Düzenlenmesi ve Erişime Açılmasına İlişkin Yönerge” gereğince tam metin erişime açılmıştır.Nesnelerin İnterneti 'nin işletmeler arasında popülerliğinin artmasıyla, izleme ve analiz IoT verilerinin araştırılması ve geliştirilmesi artmıştır. Büyük veri kaynaklarından biri olan Nesnelerin interneti, veri mühendislerinden dikkat çekiyor. Asıl zorluk, büyük miktarda IoT olayının gerçek zamanlı akış işlemesidir. Veri transferini, büyük ölçekli verileri gerçek zamanlı olarak depolamayı, işlemeyi ve analiz etmeyi içerir. Milyarlarca IoT cihazı, istihbaratı gerçek zamanda elde etmek için analiz edilmesi gereken çok miktarda veri üretir. Bu tezde, IoT için gerçek zamanlı akış işlemek için birleştirilmiş bir çözüm önerilmiştir. Önerilen yöntemde, hava istasyonu verilerinin IoT olayları Apache Kafka kullanılarak üretilir ve bir konuya yayınlanır. Bu veriler Apache Spark tüketicisi tarafından tüketilmekte ve RDD'ye dönüştürülmektedir. Spark SQL'i kullanarak, verileri analiz etmek için farklı sorguların uygulandığı veri çerçeveleri oluşturulur. Veriler Cassandra'ya kaydedilir ve Zeppelin notebook verileri görselleştirmek için kullanılır. Spark'deki makine öğrenme kütüphanesini kullanarak gerçek zamanlı tahminler yapmak için bir veri kümesine Lojistik Regresyon algoritması uygulanır. Sonunda, tüm ölçüm farklı metrikleri değiştirerek ve gecikmeyi azaltarak hızlanır. Sonuçlar, bu yöntemin gerçek zamanlı olarak büyük IoT veri kümelerini işlemek için eksiksiz bir çözüm sunduğunu göstermektedir.With the increase in popularity of IoT among enterprises, the research and development in the field of monitoring and analyzing IoT data has been increased. Iot, being one of the major sources of big data is getting attention from data engineers. The main challenge is real time stream processing of large amount of IoT events. It includes data transfer, storing, processing and analyzing large scale of data in real time. Billions of IoT devices generate huge amount of data that should be analyzed for deriving intelligence in real time. In this thesis, a unified solution for real time stream processing for IoT is proposed. In the proposed method, sample IoT events of weather station data are generated using Apache Kafka and published to a topic. This data is consumed by Apache Spark consumer which converted it into RDDs. Using Spark SQL, data frames are generated, on which different queries are applied to analyze the data. Data is saved to Cassandra and Zeppelin notebook is used to visualize the data. Logistic Regression algorithm is applied on a data set to make predictions in real time using machine learning library in Spark. In the end, the whole method is speed up by altering different metrics and reducing delay. Results show that this method provides a complete solution to process large IoT data sets in real time

Software DSM protocols that adapt between single writer and multiple writer

We present two software DSM protocols that dynamically adapt between a single writer (SW) and a multiple writer (MW) protocol based on the application's sharing patterns. The first protocol (WFS) adapts based on write-write false sharing; the second (WFS+WG) based on a combination of write-write false sharing and write granularity. The adaptation is automatic. No user or compiler information is needed. The choice between SW and MW is made on a per-page basis. We measured the performance of our adaptive protocols on an 8-node SPARC cluster connected by a 155 Mbps ATM network. We used eight applications, covering a broad spectrum in terms of write-write false sharing and write granularity. We compare our adaptive protocols against the MW-only and the SW-only approach. Adaptation to write-write false sharing proves to be the critical performance factor, while adaptation to write granularity plays only a secondary role in our environment and for the applications considered. Each of the two adaptive protocols matches or exceeds the performance of the best of MW and SW in seven out of the eight application

Improving Fine-Grained Irregular Shared-Memory Benchmarks by Data Reordering

We demonstrate that data reordering can substantially improve the performance of fine-grained irregular sharedmemory benchmarks, on both hardware and software shared-memory systems. In particular, we evaluate two distinct data reordering techniques that seek to co-locate in memory objects that are in close proximity in the physical system modeled by the computation. The effects of these techniques are increased spatial locality and reduced false sharing. We evaluate the effectiveness of the data reordering techniques on a set of five irregular applications from SPLASH-2 and Chaos. We implement both techniques in a small library, allowing us to enable them in an application by adding less than 10 lines of code. Our results on one hardware and two software shared-memory systems show that, with data reordering during initialization, the performance of these applications is improved by 12%–99% on the Origin 2000, 30%–366% on TreadMarks, and 14%–269% on HLRC

Evolutionary Computation 2020

Intelligent optimization is based on the mechanism of computational intelligence to refine a suitable feature model, design an effective optimization algorithm, and then to obtain an optimal or satisfactory solution to a complex problem. Intelligent algorithms are key tools to ensure global optimization quality, fast optimization efficiency and robust optimization performance. Intelligent optimization algorithms have been studied by many researchers, leading to improvements in the performance of algorithms such as the evolutionary algorithm, whale optimization algorithm, differential evolution algorithm, and particle swarm optimization. Studies in this arena have also resulted in breakthroughs in solving complex problems including the green shop scheduling problem, the severe nonlinear problem in one-dimensional geodesic electromagnetic inversion, error and bug finding problem in software, the 0-1 backpack problem, traveler problem, and logistics distribution center siting problem. The editors are confident that this book can open a new avenue for further improvement and discoveries in the area of intelligent algorithms. The book is a valuable resource for researchers interested in understanding the principles and design of intelligent algorithms

Adaptive Protocols for Software Distributed Shared Memory

We demonstrate the benefits of software shared memory protocols that adapt at run-time to the memory access patterns observed in the applications. This adaptation is automatic, no user annotations are required, and does not rely on compiler support or special hardware. We investigate adaptation between single- and multiple-writer protocols, dynamic aggregation of pages into a larger transfer unit, and adaptation between invalidate and update. Our results indicate that adaptation between single, and multiple-writer and dynamic page aggregation are clearly beneficial. The results for the adaptation between invalid-date and update are less compelling, showing at best gains similar to the dynamic aggregation adaptation and at worst serious performance deterioration

Compiler and Runtime Optimizations for Fine-Grained Distributed Shared Memory Systems

Bal, H.E. [Promotor

BodyCloud: a SaaS approach for community body sensor networks

Body Sensor Networks (BSNs) have been recently introduced for the remote monitoring of human activities in a broad range of application domains, such as health care, emergency management, fitness and behaviour surveillance. BSNs can be deployed in a community of people and can generate large amounts of contextual data that require a scalable approach for storage, processing and analysis. Cloud computing can provide a flexible storage and processing infrastructure to perform both online and offline analysis of data streams generated in BSNs. This paper proposes BodyCloud, a SaaS approach for community BSNs that supports the development and deployment of Cloud-assisted BSN applications. BodyCloud is a multi-tier application-level architecture that integrates a Cloud computing platform and BSN data streams middleware. BodyCloud provides programming abstractions that allow the rapid development of community BSN applications. This work describes the general architecture of the proposed approach and presents a case study for the real-time monitoring and analysis of cardiac data streams of many individuals