BIG DATA ANALYTICS - AN OVERVIEW

   Big Data Analytics has been in advance more attention recently since researchers in business and academic world are trying to successfully mine and use all possible knowledge from the vast amount of data generated and obtained. Demanding a paradigm shift in the storage, processing and analysis of Big Data, traditional data analysis methods stumble upon large amounts of data in a short period of time. Because of its importance, the U.S. Many agencies, including the government, have in recent years released large funds for research in Big Data and related fields. This gives a concise summary of investigate growth in various areas related to big data processing and analysis and terminate with a discussion of research guidelines in the similar areas. &nbsp

Seer: Empowering Software Defined Networking with Data Analytics

Network complexity is increasing, making network control and orchestration a challenging task. The proliferation of network information and tools for data analytics can provide an important insight into resource provisioning and optimisation. The network knowledge incorporated in software defined networking can facilitate the knowledge driven control, leveraging the network programmability. We present Seer: a flexible, highly configurable data analytics platform for network intelligence based on software defined networking and big data principles. Seer combines a computational engine with a distributed messaging system to provide a scalable, fault tolerant and real-time platform for knowledge extraction. Our first prototype uses Apache Spark for streaming analytics and open network operating system (ONOS) controller to program a network in real-time. The first application we developed aims to predict the mobility pattern of mobile devices inside a smart city environment.Comment: 8 pages, 6 figures, Big data, data analytics, data mining, knowledge centric networking (KCN), software defined networking (SDN), Seer, 2016 15th International Conference on Ubiquitous Computing and Communications and 2016 International Symposium on Cyberspace and Security (IUCC-CSS 2016

Аппаратно-программная инфраструктура для реализации учебных проектов по тематике обработки потоков данных в высоконагруженных информационных системах

В работе анализируется современное состояние аппаратно-программных средств обработки интенсивных потоков данных в реальном времени. Рассмотрены проблемы построения высокопроизводительного вычислительного комплекса обработки потоков данных на основе технологии IBM BigInsights. Демонстрируются возможности программного пакета Apache Spark для обработки временных рядов. Приведены примеры получения некоторых статистических характеристик временных рядов в реальном времени

The Power of Exogenous Variables in Predicting West Nile Virus in South Carolina

Despite the availability of medical data, environmental surveillance tools, and heightened public awareness, West Nile Virus (WNv) remains a global health hazard. Reliable methods for predicting WNv outbreaks remain elusive, and environmental health managers must take preventive actions without the benefit of simple predictive tools. The purpose of this ex post facto research was to examine the accuracy and timeliness of exogenous data in predicting outbreaks of WNv in South Carolina. Decision theory, the CYNEFIN construct, and systems theory provided the theoretical framework for this study, allowing the researcher to broaden traditional decision theory concepts with powerful system-level precepts. Using WNv as an example of decision making in complex environments, a statistical model for predicting the likelihood of the presence of WNv was developed through the exclusive use of exogenous explanatory variables (EEVs). The key research questions were focused on whether EEVs alone can predict the likelihood of WNv presence with the statistical confidence to make timely preventive resource decisions. Results indicated that publicly accessible EEVs such as average temperature, average wind speed, and average population can be used to predict the presence of WNv in a South Carolina locality 30 days prior to an incident, although they did not accurately predict incident counts higher than four. The social implications of this research can be far-reaching. The ability to predict emerging infectious diseases (EID) for which there are no vaccines or cure can provide decision makers with the ability to take pro-active measures to mitigate EID outbreaks

A framework for grain commodity trading decision support in South Africa

In several countries around the world, grain commodities are traded as assets on stock exchanges. This indicate that the market and effectively the prices of the grain commodities in such countries, are controlled by several local and international economic, political and social factors that are rapidly changing. As a result, the prices of some grain commodities are volatile and trading in such commodities are prone to price-related risks. There are different trading strategies for minimising price-related risks and maximising profits. But empirical research suggests that making the right decision for effective grain commodities trading has been a difficult task for stakeholders due to high volatility of grain commodities prices. Studies have shown that this is more challenging among grain commodities farmers because of their lack of skills and the time to sift through and make sense of the datasets on the plethora of factors that influence the grain commodities market. This thesis focused on providing an answer for the main research problem that grain farmers in South Africa do not take full advantage of all the available strategies for trading their grain commodities because of the complexities associated with monitoring the large datasets that influence the grain commodities market. The main objective set by this study is to design a framework that can be followed to collect, integrate and analyse datasets that influence trading decisions of grain farmers in South Africa about grain commodities. This study takes advantage of the developments in Big Data and Data Science to achieve the set objective using the Design Science Research (DSR) methodology. The prediction of future prices of grain commodities for the different trading strategies was identified as an important factor for making better decisions when trading grain commodities and the key factors that influence the prices were identified. This was followed by a critical review of the literature to determine how the concepts of Big Data and Data Science can be leveraged for an effective grain commodities trading decision support. This resulted in a proposed framework for grain commodities trading. The proposed framework suggested an investigation of the factors that influence the prices of grain commodities as the basis for acquiring the relevant datasets. The proposed framework suggested the adoption of the Big Data approach in acquiring, preparing and integrating relevant datasets from several sources. Furthermore, it was suggested that algorithmic models for predicting grain commodities prices can be developed on top of the data layer of the proposed framework to provide real-time decision support. The proposed framework suggests the need for a carefully designed visualisation of the result and the collected data that promotes user experience. Lastly, the proposed framework included a technology consideration component to support the Big Data and Data Science approach of the framework. To demonstrate that the proposed framework addressed the main problem of this research, datasets from several sources on trading white maize in South Africa and the factors that influence market were streamed, integrated and analysed. Backpropagation Neural Network algorithm was used for modelling the prices of white maize for spot and futures trading strategies were predicted. There are other modelling techniques such as the Box-Jenkins statistical time series analysis methodology. But, Neural Networks was identified as more suitable for time series data with complex patterns and relationships. A demonstration system was setup to provide effective decision support by using near real-time data to provide a dynamic predictive analytics for the spot and December futures contract prices of white maize in South Africa. Comparative analysis of predictions made using the model from the proposed framework to actual data indicated a significant degree of accuracy. A further evaluation was carried out by asking experienced traders to make predictions for the spot and December futures contract prices of white maize. The result of the exercise indicated that the predictions from the developed model were much closer to the actual prices. This indicated that the proposed framework is technically capable and generally useful. It also shows that the proposed framework can be used to provide decision support about trading grain commodities to stakeholders with lesser skills, experience and resources. The practical contribution of this thesis is that relevant datasets from several sources can be streamed into an integrated data source in real-time, which can be used as input for a real-time learning algorithmic model for predicting grain commodities prices. This will make it possible for a predictive analytics that responds to market volatility thereby providing an effective decision support for grain commodities trading. Another practical contribution of this thesis is a proposed framework that can be followed for developing a Decision Support System for trading in grain commodities. This thesis made theoretical contributions by building on the information processing theory and the decision making theory. The theoretical contribution of this thesis consists of the identification of Big Data approach, tools and techniques for eradicating uncertainty and equivocality in grain commodities trading decision making process