Data location aware scheduling for virtual Hadoop cluster deployment on private cloud computing environment

With the advancements of Internet-of-Things (IoT) and Machine-to-Machine Communications (M2M), the ability to generate massive amount of streaming data from sensory devices in distributed environment is inevitable. A common practice nowadays is to process these data in a high-performance computing infrastructure, such as cloud. Cloud platform has the ability to deploy Hadoop ecosystem on virtual clusters. In cloud configuration with different geographical regions, virtual machines (VMs) that are part of virtual cluster are placed randomly. Prior to processing, data have to be transferred to the regional sites with VMs for data locality purposes. In this paper, a provisioning strategy with data-location aware deployment for virtual cluster will be proposed, as to localize and provision the cluster near to the storage. The proposed mechanism reduces the network distance between virtual cluster and storage, resulting in reduced job completion times

Simulation framework for connected vehicles: a scoping review [version 2; peer review: 2 approved]

Background: V2V (Vehicle-to-Vehicle) is a booming research field with a diverse set of services and applications. Most researchers rely on vehicular simulation tools to model traffic and road conditions and evaluate the performance of network protocols. We conducted a scoping review to consider simulators that have been reported in the literature based on successful implementation of V2V systems, tutorials, documentation, examples, and/or discussion groups. Methods: Simulators that have limited information were not included. The selected simulators are described individually and compared based on their requirements and features, i.e., origin, traffic model, scalability, and traffic features. This scoping review was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR). The review considered only research published in English (in journals and conference papers) completed after 2015. Further, three reviewers initiated the data extraction phase to retrieve information from the published papers. Results: Most simulators can simulate system behaviour by modelling the events according to pre-defined scenarios. However, the main challenge faced is integrating the three components to simulate a road environment in either microscopic, macroscopic or mesoscopic models. These components include mobility generators, VANET simulators and network simulators. These simulators require the integration and synchronisation of the transportation domain and the communication domain. Simulation modelling can be run using a different types of simulators that are cost-effective and scalable for evaluating the performance of V2V systems in urban environments. In addition, we also considered the ability of the vehicular simulation tools to support wireless sensors. Conclusions: The outcome of this study may reduce the time required for other researchers to work on other applications involving V2V systems and as a reference for the study and development of new traffic simulators

Biologically inspired mobile agent-based sensor network (BIMAS)

Sensor nodes deployed in a large topology require autonomous capabilities and scalability since frequent replacement of the nodes is almost impossible. These sensors should be self–healing and energy efficient. Biologically inspired algorithms offer a new paradigm in providing solutions to problems found within the wireless sensor networks (WSNs). In this paper, bee nectar harvesting (analogous to data harvesting from sensor nodes) and flower pollination (analogous to energy harvesting in sensor nodes) are proposed with detailed mapping. Simulation was conducted to evaluate the proposed mechanism and a prototype was developed to show the feasibility of mobile agent deployment and energy provisioning. The research outcome is a biologically inspired mobile agent–based system (BIMAS) that provides a novel self–healing (bee pollination analogy for energy efficiency) protocol, leading to a longer WSNs lifetime. BIMAS is aimed for delay tolerant applications and where sensor node replacement is almost impossible

DHGN network with mode-based receptive fields for 2-dimensional binary pattern recognition

We introduce an extension to existing Distributed Hierarchical Graph Neuron (DHGN) network for 2-dimensional binary pattern recognition. The new form of DHGN network, termed as receptive field DHGN network (RF-DHGN) is a hybrid of a receptive field layer for 2D feature extraction, and one or more DHGN subnets for feature recognition. All inputs to the network, in the form of synaptic weights are automatically determined through mode-based activation function within the RF neurons. The proposed scheme minimizes the need for large number of neurons as compared to the normal DHGN scheme. Furthermore, the results of preliminary recognition tests indicate high recognition accuracy, similar to existing DHGN approach for distributed pattern recognition

A combined pattern recognition scheme with genetic algorithms for robot guidance using Wireless Sensor Networks

In Wireless Sensor Networks (WSNs), using physically sensed data for accurate automated decision making is challenging. In response to these challenges, a combined Genetic Algorithm (GA) and pattern recognition scheme (PR) is presented in this paper. The aim of the scheme is to reduce the exponential relationship between problem size and time complexity of GA for guiding robots using WSN. The PR scheme presented in this paper is called Cellular Weighted Pattern Recogniser (CWPR) that simplifies computations and communications for energy conservation and speeds up recognition by leveraging the parallel distributed processing capabilities of WSN. Additionally, CWPR solves the problem of dilation, translation, and rotation to provide efficient pattern recognition in energy constrained WSN environments. Combining CWPR with GA allows GA to learn from experience and solve similar problems in fewer number of generations. The experimental results show that the approach efficiently supports a variety of PR applications for WSN guided robots

Mobile data collection in Wireless Sensor Network

Battery powered Wireless Sensor Networks (WSNs) provide critical solutions to a wide range of applications including environmental monitoring, wildlife management, human and object tracking, and surveillance systems. Recharging or replacing batteries is often difficult since sensors are often placed in areas which are difficult to access. Hence this technology requires sensor nodes to be as autonomic as possible. Moreover, multihop routing in WSN causes routing holes and shorter network life time. Biologically-inspired algorithms offer a new paradigm for naturally inspired solutions to problems arising in WSN. Ant routing, bee colonization and bee optimization algorithms have shown outstanding performance for WSN. Most of these bio-inspired algorithms are applied into autonomous networking for self-organization, self-healing, self-management, and others. In this paper, data harvesting from sensor nodes and energy provision in sensor nodes derived from the analogies of bee nectar harvesting and pollination respectively, are proposed with detailed mapping. Simulation and prototype results reveal that the bio-inspired mechanism can be a potential solution

Recognising patterns in large data sets: a distributed approach

Advancements in computer architecture, high speed networks, and sensor/data capture technologies have the potential to generate vast amounts of information and bring in new forms of data processing. Unlike the early computations that worked with small chunks of data, contemporary computing infrastructure is able to generate and store large - petabytes - of data for day-to-day operations. These data may arise from high-dimensional images used in medical diagnosis to millions of multi-sensor data collected for the detection of natural events, these large-scale and complex data are increasingly becoming a common phenomenon. This poses a question of whether our ability to recognise and process these data, matches our ability to generate them. This question will be addressed, by looking at the capability of existing recognition schemes to scale up with this outgrowth of data. A different perspective is needed tomeet the challenges posed by the so called data deluge. So this thesis take a view which is somewhat outside the conventional approaches, such as statistical computations and deterministic learning schemes, this research considers the bringing together strengths of high performance and parallel computing to artificial intelligence and machine learning and thus proposes a distributed processing approach for scalable pattern recognition. The research has identified two important issues related to scalability in pattern recognition. These are complexity of learning algorithm and dependency on single processing (CPU-centric) scheme. Scalability in regards to pattern recognition, can be defined as the growth in the capability of pattern recognition algorithms to process large-scale data sets rapidly and with an acceptable level of accuracy. To scale up the recognition process, a pattern recognition system should acquire simple learning mechanisms and the ability to parallelise and distribute its processes for analysis of increasingly large and complex patterns. This thesis describes a new form of pattern recognition by enabling recognition procedure to be synthesised into a large number of loosely-coupled processes, using a fast single-cycle learning associative memory algorithm. This algorithm implements a divide-and-distribute approach on patterns, hence reducing the processing load capacity per compute node. By using this algorithm, patterns arising from diverse sources e.g. high resolution images and sensor readings may be distributed across parallel computational networks for recognition purposes using a generic framework. Furthermore, the approach enables the recognition process to be scaled up for increasing size and dimension of patterns, given sufficient processing capacity available in hand. Apart from this, a single-cycle learning mechanism being applied in this scheme allows recognition to be performed in a fast and responsive manner, without affecting the level of accuracy of the recogniser. The learning mechanism enables memorisation of a pattern within a single pass, therefore, adding more patterns to the scheme does not affect its performance and accuracy. A series of tests have been performed on recognition accuracy and computational complexity using different types of patterns ranging from facial images to sensor readings. This was done to study the accuracy and scalability of the distributed pattern recognition scheme. The results of these analyses have indicated that the proposed scheme is highly scalable, enables fast/online learning, and is able to achieve accuracy that is comparable to well known machine learning techniques. After addressing the scalability and performance aspects, this thesis deals with pattern complexity by including pattern recognition applications with multiple features. With the recognition process implemented in a distributed manner, the capacity for allowing more features to be added is possible. The proposed multi-feature approach provides an effective scheme that is capable to accommodate multiple pattern features within the analysis process. This is essential in data mining applications that involve complex data, such as biomedical images containing numerous features. The distributed multi-feature approach using single-cycle learning algorithm demonstrates high recall accuracy in the recognition simulations involving complex images. Finally, this thesis investigates the scheme's adaptability to different levels of network granularity and discovers important factors for the scalability of the pattern recognition scheme. This allows the recognition scheme to be deployed in different network conditions, ranging from coarse-grained networks such as computational grids, to fine-grained systems, including wireless sensor networks (WSNs). By acquiring resource-awareness, the proposed distributed pattern recogniser can be deployed in different kinds of applications on different network platforms, creating a generic scheme for pattern recognition. Further analysis on adaptive network granularity feature of distributed single-cycle learning pattern recognition scheme was conducted as a case study to examine the effectiveness and efficiency of the proposed approach for distributed event detection within fine-grained WSN networks. The outcomes of the study indicate that the distributed pattern recognition approach is well-suited for performing event detection using the divide-and-distribute approach with the in-network parallel processing mechanism within a resource-constrained environment. Furthermore, the ability to perform recognition using a simple learning mechanism, enables each sensor node to perform complex applications such as event detection. As a result, this research may give a new insight for applications involving large-scale event detection including forest-fire detection and structural health monitoring (SHM) for mega-structures

Recognising patterns in large data sets: a distributed approach

Advancements in computer architecture, high speed networks, and sensor/data capture technologies have the potential to generate vast amounts of information and bring in new forms of data processing. Unlike the early computations that worked with small chunks of data, contemporary computing infrastructure is able to generate and store large - petabytes - of data for day-to-day operations. These data may arise from high-dimensional images used in medical diagnosis to millions of multi-sensor data collected for the detection of natural events, these large-scale and complex data are increasingly becoming a common phenomenon. This poses a question of whether our ability to recognise and process these data, matches our ability to generate them. This question will be addressed, by looking at the capability of existing recognition schemes to scale up with this outgrowth of data. A different perspective is needed tomeet the challenges posed by the so called data deluge. So this thesis take a view which is somewhat outside the conventional approaches, such as statistical computations and deterministic learning schemes, this research considers the bringing together strengths of high performance and parallel computing to artificial intelligence and machine learning and thus proposes a distributed processing approach for scalable pattern recognition. The research has identified two important issues related to scalability in pattern recognition. These are complexity of learning algorithm and dependency on single processing (CPU-centric) scheme. Scalability in regards to pattern recognition, can be defined as the growth in the capability of pattern recognition algorithms to process large-scale data sets rapidly and with an acceptable level of accuracy. To scale up the recognition process, a pattern recognition system should acquire simple learning mechanisms and the ability to parallelise and distribute its processes for analysis of increasingly large and complex patterns. This thesis describes a new form of pattern recognition by enabling recognition procedure to be synthesised into a large number of loosely-coupled processes, using a fast single-cycle learning associative memory algorithm. This algorithm implements a divide-and-distribute approach on patterns, hence reducing the processing load capacity per compute node. By using this algorithm, patterns arising from diverse sources e.g. high resolution images and sensor readings may be distributed across parallel computational networks for recognition purposes using a generic framework. Furthermore, the approach enables the recognition process to be scaled up for increasing size and dimension of patterns, given sufficient processing capacity available in hand. Apart from this, a single-cycle learning mechanism being applied in this scheme allows recognition to be performed in a fast and responsive manner, without affecting the level of accuracy of the recogniser. The learning mechanism enables memorisation of a pattern within a single pass, therefore, adding more patterns to the scheme does not affect its performance and accuracy. A series of tests have been performed on recognition accuracy and computational complexity using different types of patterns ranging from facial images to sensor readings. This was done to study the accuracy and scalability of the distributed pattern recognition scheme. The results of these analyses have indicated that the proposed scheme is highly scalable, enables fast/online learning, and is able to achieve accuracy that is comparable to well known machine learning techniques. After addressing the scalability and performance aspects, this thesis deals with pattern complexity by including pattern recognition applications with multiple features. With the recognition process implemented in a distributed manner, the capacity for allowing more features to be added is possible. The proposed multi-feature approach provides an effective scheme that is capable to accommodate multiple pattern features within the analysis process. This is essential in data mining applications that involve complex data, such as biomedical images containing numerous features. The distributed multi-feature approach using single-cycle learning algorithm demonstrates high recall accuracy in the recognition simulations involving complex images. Finally, this thesis investigates the scheme's adaptability to different levels of network granularity and discovers important factors for the scalability of the pattern recognition scheme. This allows the recognition scheme to be deployed in different network conditions, ranging from coarse-grained networks such as computational grids, to fine-grained systems, including wireless sensor networks (WSNs). By acquiring resource-awareness, the proposed distributed pattern recogniser can be deployed in different kinds of applications on different network platforms, creating a generic scheme for pattern recognition. Further analysis on adaptive network granularity feature of distributed single-cycle learning pattern recognition scheme was conducted as a case study to examine the effectiveness and efficiency of the proposed approach for distributed event detection within fine-grained WSN networks. The outcomes of the study indicate that the distributed pattern recognition approach is well-suited for performing event detection using the divide-and-distribute approach with the in-network parallel processing mechanism within a resource-constrained environment. Furthermore, the ability to perform recognition using a simple learning mechanism, enables each sensor node to perform complex applications such as event detection. As a result, this research may give a new insight for applications involving large-scale event detection including forest-fire detection and structural health monitoring (SHM) for mega-structures

One-shot classification of 2-D leaf shapes using Distributed Hierarchical Graph Neuron (DHGN) scheme with k-NN classifier

This article presents a scalable approach for classifying plant leaves using the 2-dimensional shape feature. The proposed approach integrates a distributed recognition scheme called Distributed Hierarchical Graph Neuron (DHGN) for pattern recognition and k-nearest neighbor (k-NN) for pattern classification. With increasing amount of leaves data that can be captured using existing image gathering and processing technology, the ability for any particular classification scheme to produce high recall accuracy while adapting to large-scale dataset and data features is very important. The approach presented in this paper implements a one-shot learning mechanism within a distributed processing infrastructure, enabling large-scale data to be classified efficiently. The experimental results obtained through a series of classification tests indicate that the proposed scheme is able to produce high recall accuracy and large number of perfect recalls for a given plant leaves dataset. Furthermore, the results also indicate that the recognition procedure within the DHGN distributed scheme incurs low computational complexity and minimum processing time