A Scalable Cluster-based Infrastructure for Edge-computing Services

In this paper we present a scalable and dynamic intermediary infrastruc- ture, SEcS (acronym of BScalable Edge computing Services’’), for developing and deploying advanced Edge computing services, by using a cluster of heterogeneous machines. Our goal is to address the challenges of the next-generation Internet services: scalability, high availability, fault-tolerance and robustness, as well as programmability and quick prototyping. The system is written in Java and is based on IBM’s Web Based Intermediaries (WBI) [71] developed at IBM Almaden Research Center

Data partitioning and load balancing in parallel disk systems

Parallel disk systems provide opportunities for exploiting I/O parallelism in two possible ways, namely via inter-request and intra-request parallelism. In this paper we discuss the main issues in performance tuning of such systems, namely striping and load balancing, and show their relationship to response time and throughput. We outline the main components of an intelligent file system that optimizes striping by taking into account the requirements of the applications, and performs load balancing by judicious file allocation and dynamic redistributions of the data when access patterns change. Our system uses simple but effective heuristics that incur only little overhead. We present performance experiments based on synthetic workloads and real-life traces

Autonomic Management in a Distributed Storage System

This thesis investigates the application of autonomic management to a distributed storage system. Effects on performance and resource consumption were measured in experiments, which were carried out in a local area test-bed. The experiments were conducted with components of one specific distributed storage system, but seek to be applicable to a wide range of such systems, in particular those exposed to varying conditions. The perceived characteristics of distributed storage systems depend on their configuration parameters and on various dynamic conditions. For a given set of conditions, one specific configuration may be better than another with respect to measures such as resource consumption and performance. Here, configuration parameter values were set dynamically and the results compared with a static configuration. It was hypothesised that under non-changing conditions this would allow the system to converge on a configuration that was more suitable than any that could be set a priori. Furthermore, the system could react to a change in conditions by adopting a more appropriate configuration. Autonomic management was applied to the peer-to-peer (P2P) and data retrieval components of ASA, a distributed storage system. The effects were measured experimentally for various workload and churn patterns. The management policies and mechanisms were implemented using a generic autonomic management framework developed during this work. The experimental evaluations of autonomic management show promising results, and suggest several future research topics. The findings of this thesis could be exploited in building other distributed storage systems that focus on harnessing storage on user workstations, since these are particularly likely to be exposed to varying, unpredictable conditions.Comment: PhD Thesis, University of St Andrews, 2009. Supervisor: Graham Kirb

Adaptive and secured resource management in distributed and Internet systems

The effectiveness of computer system resource management has been always determined by two major factors: (1) workload demands and management objectives, (2) the updates of the computer technology. These two factors are dynamically changing, and resource management systems must be timely adaptive to the changes. This dissertation attempts to address several important and related resource management issues.;We first study memory system utilization in centralized servers by improving memory performance of sorting algorithms, which provides fundamental understanding on memory system organizations and its performance optimizations for data-intensive workloads. to reduce different types of cache misses, we restructure the mergesort and quicksort algorithms by integrating tiling, padding, and buffering techniques and by repartitioning the data set. Our study shows substantial performance improvements from our new methods.;We have further extended the work to improve load sharing for utilizing global memory resources in distributed systems. Aiming at reducing the memory resource contention caused by page faults and I/O activities, we have developed and examined load sharing policies by considering effective usage of global memory in addition to CPU load balancing in both homogeneous and heterogeneous clusters.;Extending our research from clusters to Internet systems, we have further investigated memory and storage utilizations in Web caching systems. We have proposed several novel management schemes to restructure and decentralize the existing caching system by exploiting data locality at different levels of the global memory hierarchy and by effectively sharing data objects among the clients and their proxy caches.;Data integrity and communication anonymity issues are raised from our decentralized Web caching system design, which are also security concerns for general peer-to-peer systems. We propose an integrity protocol to ensure data integrity, and several protocols to achieve mutual communication anonymity between an information requester and a provider.;The potential impact and contributions of this dissertation are briefly stated as follows: (1) two major research topics identified in this dissertation are fundamentally important for the growth and development of information technology, and will continue to be demanding topics for a long term. (2) Our proposed cache-effective sorting methods bridge a serious gap between analytical complexity of algorithms and their execution complexity in practice due to the increasingly deep memory hierarchy in computer systems. This approach can also be used to improve memory performance at different levels of the memory hierarchy, such as I/O and file systems. (3) Our load sharing principle of giving a high priority to the requests of data accesses in memory and I/Os timely adapts the technology changes and effectively responds to the increasing demand of data-intensive applications. (4) Our proposed decentralized Web caching framework and its resource management schemes present a comprehensive case study to examine the P2P model. Our results and experiences can be used for related and further studies in distributed computing. (5) The proposed data integrity and communication anonymity protocols address limits and weaknesses of existing ones, and place a solid foundation for us to continue our work in this important area

Comparative Analysis of Active and Passive Mapping Techniques in an Internet-Based Local Area Network

Network mapping technologies allow quick and easy discovery of computer systems throughout a network. Active mapping methods, such as using nmap, capitalize on the standard stimulus-response of network systems to probe target systems. In doing so, they create extra traffic on the network, both for the initial probe and for the target system\u27s response. Passive mapping methods work opportunistically, listening for network traffic as it transits the system. As such, passive methods generate minimal network traffic overhead. Active methods are still standard methods for network information gathering; passive techniques are not normally used due to the possibility of missing important information as it passes by the sensor. Configuring the network for passive network mapping also involves more network management. This research explores the implementation of a prototype passive network mapping system, lanmap, designed for use within an Internet Protocol-based local area network. Network traffic is generated by a synthetic traffic generation suite using honeyd and syntraf, a custom Java program to interact with honeyd. lanmap is tested against nmap to compare the two techniques. Experimental results show that lanmap is quite effective, discovering an average of 76.1% of all configured services (server- and client-side) whereas nmap only found 27.6% of all configured services. Conversely, lanmap discovered 19.9% of the server services while nmap discovered 92.7% of the configured server-side services. lanmap discovered 100% of all client-side service consumers while nmap found none. lanmap generated an average of 200 packets of network overhead while nmap generated a minimum of minimum 8,600 packets on average?up to 155,000 packets at its maximum average value. The results show that given the constraints of the test bed, passive network mapping is a viable alternative to action network mapping, unless the mapper is looking for server-side services

The Architecture of an Autonomic, Resource-Aware, Workstation-Based Distributed Database System

Distributed software systems that are designed to run over workstation machines within organisations are termed workstation-based. Workstation-based systems are characterised by dynamically changing sets of machines that are used primarily for other, user-centric tasks. They must be able to adapt to and utilize spare capacity when and where it is available, and ensure that the non-availability of an individual machine does not affect the availability of the system. This thesis focuses on the requirements and design of a workstation-based database system, which is motivated by an analysis of existing database architectures that are typically run over static, specially provisioned sets of machines. A typical clustered database system -- one that is run over a number of specially provisioned machines -- executes queries interactively, returning a synchronous response to applications, with its data made durable and resilient to the failure of machines. There are no existing workstation-based databases. Furthermore, other workstation-based systems do not attempt to achieve the requirements of interactivity and durability, because they are typically used to execute asynchronous batch processing jobs that tolerate data loss -- results can be re-computed. These systems use external servers to store the final results of computations rather than workstation machines. This thesis describes the design and implementation of a workstation-based database system and investigates its viability by evaluating its performance against existing clustered database systems and testing its availability during machine failures.Comment: Ph.D. Thesi

An optimized Load Balancing Technique for Virtual Machine Migration in Cloud Computing

Cloud computing (CC) is a service that uses subscription storage & computing power. Load balancing in distributed systems is one of the most critical pieces. CC has been a very interesting and important area of research because CC is one of the best systems that stores data with reduced costs and can be viewed over the internet at all times. Load balance facilitates maintaining high user retention & resource utilization by ensuring that each computing resource is correctly and properly distributed. This paper describes cloud-based load balancing systems. CC is virtualization of hardware like storage, computing, and security by virtual machines (VM). The live relocation of these machines provides many advantages, including high availability, hardware repair, fault tolerance, or workload balancing. In addition to various VM migration facilities, during the migration process, it is subject to significant security risks which the industry hesitates to accept. In this paper we have discussed CC besides this we also emphasize various existing load balancing algorithms, advantages& also we describe the PSO optimization technique

Tietotekniikan kehitysprojekti eräässä tutkimusorganisaatiossa

Organisaatioiden käyttämä informaatioteknologia (IT) on työntekijöiden tuottavuuden kannalta kriittinen tekijä erityisesti tutkimustyössä, joka on hyvin riippuvainen IT -infrastruktuurin toimivuudesta. Tämä diplomityö kuvaa IT -kehitysprojektin, joka toteutettiin Lääketieteellisen tekniikan ja laskennallisen tieteen laitoksella Aalto-yliopiston teknillisessä korkeakoulussa. Tutkimuksessa käytettiin toimintatutkimus-menetelmää, joka yhdistää tutkimuksen ja organisaation kehityksen käytännön ongelmatilanteessa. Tutkimuksen alussa IT -ympäristön ongelma-alueet kartoitettiin kyselyllä ja käyttäjähaastatteluilla. Organisaatiossa ilmeni useita kehitysalueita, ja toteutettavat parannustoimenpiteet priorisoitiin käyttäjien tarpeiden perusteella. Tutkimuksen laajuuden rajaamiseksi vain tärkeimmät parannustoimenpiteet valittiin toteutettavaksi tässä tutkimuksissa. Tärkeimmät toteutetut toimenpiteet tutkimuksen aikana olivat IT -palveluportfolion luominen, Linux-työasemien päivittäminen ja lähiverkon rakenteen uudistaminen. Linux-työasemien päivittämiseen kuului sopivan Linux-jakelun sekä konfiguraationhallintatyökalun valitseminen. Muita toteutettuja toimenpiteitä olivat sähköpostipalvelun siirtäminen yliopiston keskitetylle IT -organisaatiolle sekä tulostimien ja skannereiden luotettavuuden parantaminen. Toimenpiteiden vaikutusta arvioitiin toisessa kyselyssä, joka osoitti käyttäjien tyytyväisyyden parantuneen tilastollisesti merkittävästi Linux-työasemien, sähköpostipalveluiden ja IT -tuen vasteaikojen osalta. Tutkimuksen luotettavuutta arvioitiin sekä tilastollisen päättelyn että toimintatutkimuksen kriteereiden perusteella. Työssä pohditaan myös tulosten siirrettävyyttä muihin samankaltaisiin lähtötilanteisiin.An IT -environment of an organization is a critical element for the productivity of the employees, especially in research work which is very dependent on IT -infrastructure. This Master's Thesis presents an IT improvement project that was carried out at the Department of Biomedical Engineering and Computational Science (BECS) at Aalto University School of Science and Technology. The project was carried out using Action Research -methodology, that combines the goals of conducting scientific research and improving subjects in a practical situation. The improvement areas were investigated using a survey and user interviews. Several development areas were identified and improvement actions to solve the problems were prioritized according to the user needs. Major improvements implemented during the project include an IT service portfolio for managing the IT services, upgrading the Linux-workstations and renewing the structure of the network. Upgrading the Linux-workstations also included selecting a suitable Linux distribution and a tool for centralized configuration management. Other improvement actions include migrating the email service to a centralized system and improving printer and scanner reliability. The results of the actions were evaluated in a follow-up survey, which showed statistically significant increases in user satisfaction for Linux-workstations, email services and response times for IT -support. The validity of the research is analyzed according to both statistical and Action Research criteria. Also the transferability of the results to other similar situations is discussed