A Dynamic Scaling Methodology for Improving Performance of Big Data Systems

The continuous growth of data volume in various fields such as, healthcare, sciences, economics, and business has caused an overwhelming flow of data in the last decade. The overwhelming flow of data has raised challenges in processing, analyzing, and storing data, which lead many systems to face an issue in performance. Poor performance of systems creates negative impact such as delays, unprocessed data, and increasing response time. Processing huge amounts of data demands a powerful computational infrastructure to ensure that data processing and analysis success [7]. However, the architectures of these systems are not suitable to process that quantity of data. This calls for necessity to develop a methodology to improve the performance of systems handle massive amount of data. This thesis presents a novel dynamic scaling methodology to improve the performance of big data systems. The dynamic scaling methodology is developed to scale up the system based on the several aspects from the big data perspective. Moreover, these aspects are used by the helper project algorithm which is designed to divide a task into small chunks to be processed by the system. These small chunks run on several virtual machines to work in parallel to enhance the system’s runtime performance. In addition, the dynamic scaling methodology does not require many modifications on the applied, which makes it easy to use. The dynamic scaling methodology improves the performance of the big data system significantly. As a result, it provides a solution for performance failures in systems that process huge amount of data. This is study would be beneficial to IT researches that focus on performance of big data systems

Service Integration Design Patterns in Microservices

“Microservices” is a new term in software architecture that was defined in 2014 [1]. It is a method to build a software application with a set of small services. Each service has its process to serve a single purpose and communicates with other services through lightweight mechanisms. Because of a great deal of independently distributed services, it is a challenge to integrate the loose services fully. Too many trivial relationships can be messed up easily during deployment. Also, it is hard to modify the relationships if the services are updated as the source codes need to be re-edited and tested. The microservices architecture is attracting much attention recently. More and more software-developers are producing continuous applications and microservices deliveries [2]. There is a need to develop a mechanism to better integrate the scattered services during the application delivery process. The thesis proposes three general design patterns to integrate services in microservices architecture. These patterns are classified by the inter-service communication mechanisms and described with specific problems, contexts, solutions, example implementations and consequences. Besides, the informative guidelines are provided to make these patterns apply in different applications quickly. The service integration design patterns help compose services and facilitate the process of building applications in microservices. All the patterns are helpful tools to address the service integration issues in microservices. Each approach is simple and flexible to apply generally. The structures can be easily modified through these approaches

Integrating Malaria Surveillance with Climate Data for Outbreak Detection and Forecasting: the EPIDEMIA System

Background: Early indication of an emerging malaria epidemic can provide an opportunity for proactive interventions. Challenges to the identification of nascent malaria epidemics include obtaining recent epidemiological surveillance data, spatially and temporally harmonizing this information with timely data on environmental precursors, applying models for early detection and early warning, and communicating results to public health officials. Automated web-based informatics systems can provide a solution to these problems, but their implementation in real-world settings has been limited. Methods: The Epidemic Prognosis Incorporating Disease and Environmental Monitoring for Integrated Assessment (EPIDEMIA) computer system was designed and implemented to integrate disease surveillance with environmental monitoring in support of operational malaria forecasting in the Amhara region of Ethiopia. A co-design workshop was held with computer scientists, epidemiological modelers, and public health partners to develop an initial list of system requirements. Subsequent updates to the system were based on feedback obtained from system evaluation workshops and assessments conducted by a steering committee of users in the public health sector.Results: The system integrated epidemiological data uploaded weekly by the Amhara Regional Health Bureau with remotely-sensed environmental data freely available from online archives. Environmental data were acquired and processed automatically by the EASTWeb software program. Additional software was developed to implement a public health interface for data upload and download, harmonize the epidemiological and environmental data into a unified database, automatically update time series forecasting models, and generate formatted reports. Reporting features included district-level control charts and maps summarizing epidemiological indicators of emerging malaria outbreaks, environmental risk factors, and forecasts of future malaria risk. Conclusions: Successful implementation and use of EPIDEMIA is an important step forward in the use of epidemiological and environmental informatics systems for malaria surveillance. Developing software to automate the workflow steps while remaining robust to continual changes in the input data streams was a key technical challenge. Continual stakeholder involvement throughout design, implementation, and operation has created a strong enabling environment that will facilitate the ongoing development, application, and testing of the system