Replica Creation Algorithm for Data Grids

Data grid system is a data management infrastructure that facilitates reliable access and sharing of large amount of data, storage resources, and data transfer services that can be scaled across distributed locations. This thesis presents a new replication algorithm that improves data access performance in data grids by distributing relevant data copies around the grid. The new Data Replica Creation Algorithm (DRCM) improves performance of data grid systems by reducing job execution time and making the best use of data grid resources (network bandwidth and storage space). Current algorithms focus on number of accesses in deciding which file to replicate and where to place them, which ignores resources’ capabilities. DRCM differs by considering both user and resource perspectives; strategically placing replicas at locations that provide the lowest transfer cost. The proposed algorithm uses three strategies: Replica Creation and Deletion Strategy (RCDS), Replica Placement Strategy (RPS), and Replica Replacement Strategy (RRS). DRCM was evaluated using network simulation (OptorSim) based on selected performance metrics (mean job execution time, efficient network usage, average storage usage, and computing element usage), scenarios, and topologies. Results revealed better job execution time with lower resource consumption than existing approaches. This research contributes replication strategies embodied in one algorithm that enhances data grid performance, capable of making a decision on creating or deleting more than one file during same decision. Furthermore, dependency-level-between-files criterion was utilized and integrated with the exponential growth/decay model to give an accurate file evaluation

Resource Management in a Peer to Peer Cloud Network for IoT

Software-Defined Internet of Things (SDIoT) is defined as merging heterogeneous objects in a form of interaction among physical and virtual entities. Large scale of data centers, heterogeneity issues and their interconnections have made the resource management a hard problem specially when there are different actors in cloud system with different needs. Resource management is a vital requirement to achieve robust networks specially with facing continuously increasing amount of heterogeneous resources and devices to the network. The goal of this paper is reviews to address IoT resource management issues in cloud computing services. We discuss the bottlenecks of cloud networks for IoT services such as mobility. We review Fog computing in IoT services to solve some of these issues. It provides a comprehensive literature review of around one hundred studies on resource management in Peer to Peer Cloud Networks and IoT. It is very important to find a robust design to efficiently manage and provision requests and available resources. We also reviewed different search methodologies to help clients find proper resources to answer their needs

Binary vote assignment on grid quorum replication technique with association rule

One of the biggest challenges that data grids users have to face today relates to the improvement of the data management. Organizations need to provide current data to users who may be geographically remote and to handle a volume of requests of data distributed around multiple sites in distributed environment. Therefore, the storage, availability, and consistency are important issues to be addressed to allow efficient and safe data access from many different sites. One way to effectively cope with these challenges is to rely on the replication technique. Replication is a useful technique for distributed database systems. Through this technique, a data can be accessed from multiple locations. Thus, replication increases data availability and accessibility to users. When one site fails, user still can access the same data at another site. Techniques such as Read-One-Write-All (ROWA), Hierarchical Replication Scheme (HRS) and Branch Replication Scheme (BRS) are the popular techniques being used for replication and data management. However, these techniques have its weaknesses in terms of communication costs that is the total replication servers needed to replicate the data. Furthermore, these techniques also do not consider the correlation between data during the fragmentation process. The knowledge about data correlation can be extracted from historical data using techniques of the data mining field. Without proper strategies, replication increases job execution time. In this research, the some-data-to-some-sites scheme called Binary Vote Assignment on Grid Quorum with Association (BV AGQAR) is proposed to manage replication for meaningful fragmented data in distributed database environment with low communication cost and processing time for a transaction. The main feature of BV AGQ-AR is that the technique integrates replication and data mining technique allowing meaningful extraction of knowledge from large data sets. Performance of the BVAGQ-AR technique comprised the following steps. First step is mining the data by using Apriori algorithm from Association Rules. It is used to discover the correlation between data. For the second step, the database is fragmented based on the data mining analysis results. This technique is executed to make sure data replication can be effectively done while saving cost. Then, the databases that are resulted after the fragmentation process are allocated at their assigned sites. Finally, after allocation process, each site has a database file and ready for any transaction and replication process. Finally, the result of the experiments shows that BV AGQ-AR can preserve the data consistency with the lowest communication cost and processing time for a transaction as compared to BCSA, PRA, ROW A, HRS and BRS

An enhanced dynamic replica creation and eviction mechanism in data grid federation environment

Data Grid Federation system is an infrastructure that connects several grid systems, which facilitates sharing of large amount of data, as well as storage and computing resources. The existing mechanisms on data replication focus on finding file values based on the number of files access in deciding which file to replicate, and place new replicas on locations that provide minimum read cost. DRCEM finds file values based on logical dependencies in deciding which file to replicate, and allocates new replicas on locations that provide minimum replica placement cost. This thesis presents an enhanced data replication strategy known as Dynamic Replica Creation and Eviction Mechanism (DRCEM) that utilizes the usage of data grid resources, by allocating appropriate replica sites around the federation. The proposed mechanism uses three schemes: 1) Dynamic Replica Evaluation and Creation Scheme, 2) Replica Placement Scheme, and 3) Dynamic Replica Eviction Scheme. DRCEM was evaluated using OptorSim network simulator based on four performance metrics: 1) Jobs Completion Times, 2) Effective Network Usage, 3) Storage Element Usage, and 4) Computing Element Usage. DRCEM outperforms ELALW and DRCM mechanisms by 30% and 26%, in terms of Jobs Completion Times. In addition, DRCEM consumes less storage compared to ELALW and DRCM by 42% and 40%. However, DRCEM shows lower performance compared to existing mechanisms regarding Computing Element Usage, due to additional computations of files logical dependencies. Results revealed better jobs completion times with lower resource consumption than existing approaches. This research produces three replication schemes embodied in one mechanism that enhances the performance of Data Grid Federation environment. This has contributed to the enhancement of the existing mechanism, which is capable of deciding to either create or evict more than one file during a particular time. Furthermore, files logical dependencies were integrated into the replica creation scheme to evaluate data files more accurately

D^2PS: A Dependable Data Provisioning Service in Multi-tenant Cloud Environment

Software as a Service (SaaS) is a software delivery and business model widely used by Cloud computing. Instead of purchasing and maintaining a software suite permanently, customers only need to lease the software on-demand. The domain of high assurance distributed systems has focused greatly on the areas of fault tolerance and dependability. In a multi-tenant context, it is particularly important to store, manage and provision data services to customers in a highly efficient and dependable manner due to a large number of file operations involved in running such services. It is also desirable to allow a user group to share and cooperate (e.g., co-edit) on some specific data. In this paper we present a dependable data provisioning service in a multi-tenant Cloud environment. We describe a metadata management approach and leverage multiple replicated metadata caching to shorten the file access time, with the improved efficiency of data sharing. In order to reduce frequent data transmission and data access latency, we introduce a distributed cooperative disk cache mechanism that supports effective cache placement and pull-push cache synchronization. In addition, we use efficient component failover to enhance the service dependability whilst avoiding negative impact from system failures. Our experimental results show that our system can significantly reduce both unused data transmission and response latency. Specifically, over 50% network transmission and operational latency can be saved for random reads while 28.24% network traffic and 25% response latency can be reduced for random write operations. We believe that these findings are demonstrating positive results along the right direction of resolving storage-related challenges in a multi-tenant Cloud environment

University Blockchain Research Initiative (UBRI): Boosting blockchain education and research

Since its conceptualization, blockchain technology has witnessed continuous and rapid development, bringing profound changes to computer science, law, and economics. In 2008, the initial blockchain system was merely a growing list of records linked together using cryptography. Today, however, blockchains have become the foundation of most digital currencies, robust cloud computing platforms, and dependable databases for tracking supply chain information. Despite challenges and controversies, blockchain technology has the potential to help build a trustworthy and efficient digital world

A Novel Locality Algorithm and Peer-to-Peer Communication Infrastructure for Optimizing Network Performance in Smart Microgrids

[EN] Peer-to-Peer (P2P) overlay communications networks have emerged as a new paradigm for implementing distributed services in microgrids due to their potential benefits: they are robust, scalable, fault-tolerant, and they can route messages even with a large number of nodes which are frequently entering or leaving from the network. However, current P2P systems have been mainly developed for file sharing or cycle sharing applications where the processes of searching and managing resources are not optimized. Locality algorithms have gained a lot of attention due to their potential to provide an optimized path to groups with similar interests for routing messages in order to get better network performance. This paper develops a fully functional decentralized communication architecture with a new P2P locality algorithm and a specific protocol for monitoring and control of microgrids. Experimental results show that the proposed locality algorithm reduces the number of lookup messages and the lookup delay time. Moreover, the proposed communication architecture heavily depends of the lookup used algorithm as well as the placement of the communication layers within the architecture. Experimental results will show that the proposed techniques meet the network requirements of smart microgrids even with a large number of nodes on stream.This work is supported by the Spanish Ministry of Economy and Competitiveness (MINECO) and the European Regional Development Fund (ERDF) under Grant ENE2015-64087-C2-2R. This work is supported by the Spanish Ministry of Economy and Competitiveness (MINECO) under BES-2013-064539.Marzal-Romeu, S.; González-Medina, R.; Salas-Puente, RA.; Figueres Amorós, E.; Garcerá, G. (2017). A Novel Locality Algorithm and Peer-to-Peer Communication Infrastructure for Optimizing Network Performance in Smart Microgrids. Energies. 10(9):1-25. https://doi.org/10.3390/en10091275S125109Khan, R. H., & Khan, J. Y. (2013). A comprehensive review of the application characteristics and traffic requirements of a smart grid communications network. Computer Networks, 57(3), 825-845. doi:10.1016/j.comnet.2012.11.002Dada, J. O. (2014). Towards understanding the benefits and challenges of Smart/Micro-Grid for electricity supply system in Nigeria. Renewable and Sustainable Energy Reviews, 38, 1003-1014. doi:10.1016/j.rser.2014.07.077Lidula, N. W. A., & Rajapakse, A. D. (2011). Microgrids research: A review of experimental microgrids and test systems. Renewable and Sustainable Energy Reviews, 15(1), 186-202. doi:10.1016/j.rser.2010.09.041Hussain, A., Arif, S. M., Aslam, M., & Shah, S. D. A. (2017). Optimal siting and sizing of tri-generation equipment for developing an autonomous community microgrid considering uncertainties. Sustainable Cities and Society, 32, 318-330. doi:10.1016/j.scs.2017.04.004Dehghanpour, K., Colson, C., & Nehrir, H. (2017). A Survey on Smart Agent-Based Microgrids for Resilient/Self-Healing Grids. Energies, 10(5), 620. doi:10.3390/en10050620Palizban, O., Kauhaniemi, K., & Guerrero, J. M. (2014). Microgrids in active network management – part II: System operation, power quality and protection. Renewable and Sustainable Energy Reviews, 36, 440-451. doi:10.1016/j.rser.2014.04.048Shi, W., Li, N., Chu, C.-C., & Gadh, R. (2017). Real-Time Energy Management in Microgrids. IEEE Transactions on Smart Grid, 8(1), 228-238. doi:10.1109/tsg.2015.2462294Deng, R., Yang, Z., Chow, M.-Y., & Chen, J. (2015). A Survey on Demand Response in Smart Grids: Mathematical Models and Approaches. IEEE Transactions on Industrial Informatics, 11(3), 570-582. doi:10.1109/tii.2015.2414719Moazami Goodarzi, H., & Kazemi, M. (2017). A Novel Optimal Control Method for Islanded Microgrids Based on Droop Control Using the ICA-GA Algorithm. Energies, 10(4), 485. doi:10.3390/en10040485Erol-Kantarci, M., Kantarci, B., & Mouftah, H. (2011). Reliable overlay topology design for the smart microgrid network. IEEE Network, 25(5), 38-43. doi:10.1109/mnet.2011.6033034Hassan Youssef, K. (2016). Optimal management of unbalanced smart microgrids for scheduled and unscheduled multiple transitions between grid-connected and islanded modes. Electric Power Systems Research, 141, 104-113. doi:10.1016/j.epsr.2016.07.015Giotitsas, C., Pazaitis, A., & Kostakis, V. (2015). A peer-to-peer approach to energy production. Technology in Society, 42, 28-38. doi:10.1016/j.techsoc.2015.02.002Kazmi, S. A. A., Shahzad, M. K., Khan, A. Z., & Shin, D. R. (2017). Smart Distribution Networks: A Review of Modern Distribution Concepts from a Planning Perspective. Energies, 10(4), 501. doi:10.3390/en10040501Werth, A., Andre, A., Kawamoto, D., Morita, T., Tajima, S., Tokoro, M., … Tanaka, K. (2018). Peer-to-Peer Control System for DC Microgrids. IEEE Transactions on Smart Grid, 9(4), 3667-3675. doi:10.1109/tsg.2016.2638462Deconinck, G., Vanthournout, K., Beitollahi, H., Qui, Z., Duan, R., Nauwelaers, B., … Belmans, R. (2008). A Robust Semantic Overlay Network for Microgrid Control Applications. Architecting Dependable Systems V, 101-123. doi:10.1007/978-3-540-85571-2_5Bandara, H. M. N. D., & Jayasumana, A. P. (2012). Collaborative applications over peer-to-peer systems–challenges and solutions. Peer-to-Peer Networking and Applications, 6(3), 257-276. doi:10.1007/s12083-012-0157-3Palizban, O., & Kauhaniemi, K. (2015). Hierarchical control structure in microgrids with distributed generation: Island and grid-connected mode. Renewable and Sustainable Energy Reviews, 44, 797-813. doi:10.1016/j.rser.2015.01.008Khatibzadeh, A., Besmi, M., Mahabadi, A., & Reza Haghifam, M. (2017). Multi-Agent-Based Controller for Voltage Enhancement in AC/DC Hybrid Microgrid Using Energy Storages. Energies, 10(2), 169. doi:10.3390/en10020169Planas, E., Gil-de-Muro, A., Andreu, J., Kortabarria, I., & Martínez de Alegría, I. (2013). General aspects, hierarchical controls and droop methods in microgrids: A review. Renewable and Sustainable Energy Reviews, 17, 147-159. doi:10.1016/j.rser.2012.09.032Olivares, D. E., Mehrizi-Sani, A., Etemadi, A. H., Canizares, C. A., Iravani, R., Kazerani, M., … Hatziargyriou, N. D. (2014). Trends in Microgrid Control. IEEE Transactions on Smart Grid, 5(4), 1905-1919. doi:10.1109/tsg.2013.2295514Vandoorn, T. L., Vasquez, J. C., De Kooning, J., Guerrero, J. M., & Vandevelde, L. (2013). Microgrids: Hierarchical Control and an Overview of the Control and Reserve Management Strategies. IEEE Industrial Electronics Magazine, 7(4), 42-55. doi:10.1109/mie.2013.2279306Zhou, B., Li, W., Chan, K. W., Cao, Y., Kuang, Y., Liu, X., & Wang, X. (2016). Smart home energy management systems: Concept, configurations, and scheduling strategies. Renewable and Sustainable Energy Reviews, 61, 30-40. doi:10.1016/j.rser.2016.03.047Ancillotti, E., Bruno, R., & Conti, M. (2013). The role of communication systems in smart grids: Architectures, technical solutions and research challenges. Computer Communications, 36(17-18), 1665-1697. doi:10.1016/j.comcom.2013.09.004Llaria, A., Terrasson, G., Curea, O., & Jiménez, J. (2016). Application of Wireless Sensor and Actuator Networks to Achieve Intelligent Microgrids: A Promising Approach towards a Global Smart Grid Deployment. Applied Sciences, 6(3), 61. doi:10.3390/app6030061Luna, A. C., Diaz, N. L., Graells, M., Vasquez, J. C., & Guerrero, J. M. (2016). Cooperative energy management for a cluster of households prosumers. IEEE Transactions on Consumer Electronics, 62(3), 235-242. doi:10.1109/tce.2016.7613189Gungor, V. C., Lu, B., & Hancke, G. P. (2010). Opportunities and Challenges of Wireless Sensor Networks in Smart Grid. IEEE Transactions on Industrial Electronics, 57(10), 3557-3564. doi:10.1109/tie.2009.2039455Zhao, C., He, J., Cheng, P., & Chen, J. (2017). Consensus-Based Energy Management in Smart Grid With Transmission Losses and Directed Communication. IEEE Transactions on Smart Grid, 8(5), 2049-2061. doi:10.1109/tsg.2015.2513772Lo, C.-H., & Ansari, N. (2013). Decentralized Controls and Communications for Autonomous Distribution Networks in Smart Grid. IEEE Transactions on Smart Grid, 4(1), 66-77. doi:10.1109/tsg.2012.2228282Li, C., Savaghebi, M., Guerrero, J., Coelho, E., & Vasquez, J. (2016). Operation Cost Minimization of Droop-Controlled AC Microgrids Using Multiagent-Based Distributed Control. Energies, 9(9), 717. doi:10.3390/en9090717Wu, X., Jiang, P., & Lu, J. (2014). Multiagent-Based Distributed Load Shedding for Islanded Microgrids. Energies, 7(9), 6050-6062. doi:10.3390/en7096050Kantamneni, A., Brown, L. E., Parker, G., & Weaver, W. W. (2015). Survey of multi-agent systems for microgrid control. Engineering Applications of Artificial Intelligence, 45, 192-203. doi:10.1016/j.engappai.2015.07.005Lopes, A. L., & Botelho, L. M. (2008). Improving Multi-Agent Based Resource Coordination in Peer-to-Peer Networks. Journal of Networks, 3(2). doi:10.4304/jnw.3.2.38-47Cameron, A., Stumptner, M., Nandagopal, N., Mayer, W., & Mansell, T. (2015). Rule-based peer-to-peer framework for decentralised real-time service oriented architectures. Science of Computer Programming, 97, 202-234. doi:10.1016/j.scico.2014.06.005Zhang, C., Wu, J., Cheng, M., Zhou, Y., & Long, C. (2016). A Bidding System for Peer-to-Peer Energy Trading in a Grid-connected Microgrid. Energy Procedia, 103, 147-152. doi:10.1016/j.egypro.2016.11.264Malatras, A. (2015). State-of-the-art survey on P2P overlay networks in pervasive computing environments. Journal of Network and Computer Applications, 55, 1-23. doi:10.1016/j.jnca.2015.04.014Eng Keong Lua, Crowcroft, J., Pias, M., Sharma, R., & Lim, S. (2005). A survey and comparison of peer-to-peer overlay network schemes. IEEE Communications Surveys & Tutorials, 7(2), 72-93. doi:10.1109/comst.2005.1610546Xu, J., Kumar, A., & Yu, X. (2004). On the Fundamental Tradeoffs Between Routing Table Size and Network Diameter in Peer-to-Peer Networks. IEEE Journal on Selected Areas in Communications, 22(1), 151-163. doi:10.1109/jsac.2003.818805Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., & Balakrishnan, H. (2001). Chord. ACM SIGCOMM Computer Communication Review, 31(4), 149-160. doi:10.1145/964723.383071Rowstron, A., & Druschel, P. (2001). Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems. Lecture Notes in Computer Science, 329-350. doi:10.1007/3-540-45518-3_18Yuh-Jzer Joung, Li-Wei Yang, & Chien-Tse Fang. (2007). Keyword search in DHT-based peer-to-peer networks. IEEE Journal on Selected Areas in Communications, 25(1), 46-61. doi:10.1109/jsac.2007.070106Stoica, I., Morris, R., Liben-Nowell, D., Karger, D. R., Kaashoek, M. F., Dabek, F., & Balakrishnan, H. (2003). Chord: a scalable peer-to-peer lookup protocol for internet applications. IEEE/ACM Transactions on Networking, 11(1), 17-32. doi:10.1109/tnet.2002.808407Gottron, C., König, A., & Steinmetz, R. (2010). A Survey on Security in Mobile Peer-to-Peer Architectures—Overlay-Based vs. Underlay-Based Approaches. Future Internet, 2(4), 505-532. doi:10.3390/fi2040505Seyedi, Y., Karimi, H., & Guerrero, J. M. (2017). Centralized Disturbance Detection in Smart Microgrids With Noisy and Intermittent Synchrophasor Data. IEEE Transactions on Smart Grid, 8(6), 2775-2783. doi:10.1109/tsg.2016.2539947Youssef, T., Elsayed, A., & Mohammed, O. (2016). Data Distribution Service-Based Interoperability Framework for Smart Grid Testbed Infrastructure. Energies, 9(3), 150. doi:10.3390/en9030150Liu, X., Xia, H., & Chien, A. A. (2004). Validating and Scaling the MicroGrid: A Scientific Instrument for Grid Dynamics. Journal of Grid Computing, 2(2), 141-161. doi:10.1007/s10723-004-4200-3Kansal, P., & Bose, A. (2012). Bandwidth and Latency Requirements for Smart Transmission Grid Applications. IEEE Transactions on Smart Grid, 3(3), 1344-1352. doi:10.1109/tsg.2012.2197229Kuo, M.-T., & Lu, S.-D. (2013). Design and Implementation of Real-Time Intelligent Control and Structure Based on Multi-Agent Systems in Microgrids. Energies, 6(11), 6045-6059. doi:10.3390/en6116045Del Val, E., Rebollo, M., & Botti, V. (2012). Enhancing decentralized service discovery in open service-oriented multi-agent systems. Autonomous Agents and Multi-Agent Systems, 28(1), 1-30. doi:10.1007/s10458-012-9210-0Howell, S., Rezgui, Y., Hippolyte, J.-L., Jayan, B., & Li, H. (2017). Towards the next generation of smart grids: Semantic and holonic multi-agent management of distributed energy resources. Renewable and Sustainable Energy Reviews, 77, 193-214. doi:10.1016/j.rser.2017.03.107Frey, S., Diaconescu, A., Menga, D., & Demeure, I. (2015). A Generic Holonic Control Architecture for Heterogeneous Multiscale and Multiobjective Smart Microgrids. ACM Transactions on Autonomous and Adaptive Systems, 10(2), 1-21. doi:10.1145/2700326Miers, C., Simplicio, M., Gallo, D., Carvalho, T., Bressan, G., Souza, V., … Damola, A. (2010). A Taxonomy for Locality Algorithms on Peer-to-Peer Networks. IEEE Latin America Transactions, 8(4), 323-331. doi:10.1109/tla.2010.5595121Porsinger, T., Janik, P., Leonowicz, Z., & Gono, R. (2017). Modelling and Optimization in Microgrids. Energies, 10(4), 523. doi:10.3390/en10040523Ali, M., Zakariya, M., Asif, M., & Ullah, A. (2012). TCP/IP Based Intelligent Load Management System in Micro-Grids Network Using MATLAB/Simulink. Energy and Power Engineering, 04(04), 283-289. doi:10.4236/epe.2012.44038Shin, I.-J., Song, B.-K., & Eom, D.-S. (2017). International Electronical Committee (IEC) 61850 Mapping with Constrained Application Protocol (CoAP) in Smart Grids Based European Telecommunications Standard Institute Machine-to-Machine (M2M) Environment. Energies, 10(3), 393. doi:10.3390/en10030393Loh, P. C., Li, D., Chai, Y. K., & Blaabjerg, F. (2013). Autonomous Operation of Hybrid Microgrid With AC and DC Subgrids. IEEE Transactions on Power Electronics, 28(5), 2214-2223. doi:10.1109/tpel.2012.2214792Overlay networks for smart gridshttp://users.atlantis.ugent.be/cdvelder/papers/2013/wauters2013sgv.pdfEugster, P. T., Felber, P. A., Guerraoui, R., & Kermarrec, A.-M. (2003). The many faces of publish/subscribe. ACM Computing Surveys, 35(2), 114-131. doi:10.1145/857076.857078Ali, I. (2012). High-speed Peer-to-peer Communication based Protection Scheme Implementation and Testing in Laboratory. International Journal of Computer Applications, 38(4), 16-24. doi:10.5120/4596-6793Yoo, B.-K., Yang, S.-H., Yang, H.-S., Kim, W.-Y., Jeong, Y.-S., Han, B.-M., & Jang, K.-S. (2011). Communication Architecture of the IEC 61850-based Micro Grid System. Journal of Electrical Engineering and Technology, 6(5), 605-612. doi:10.5370/jeet.2011.6.5.605Dou, X., Quan, X., Wu, Z., Hu, M., Yang, K., Yuan, J., & Wang, M. (2014). Hybrid Multi-Agent Control in Microgrids: Framework, Models and Implementations Based on IEC 61850. Energies, 8(1), 31-58. doi:10.3390/en801003