Pemanfaatan Raspberry Pi 3 dan Hadoop Sebagai Pembatas Penyimpanan Online Berbasiskan Website

Jaringan komputer saat ini, merupakan sarana untuk mengakses data di setiap ruang dan waktu melalui perangkat komputer ataupun perangkat mobile. Penyimpanan data yang terhubung ke jaringan internet diharapkan dapat memberikan layanan yang optimal ketika terjadinya perpindahan data, proses upload ataupun proses download data ke penyimpanan data melaui perangkat yang digunakan oleh user. Ketika user semakin banyak, hal ini dapat menimbulkan kebanjiran data yang akan disimpan dalam perangkat penyimpanan. Selain itu, semakin banyak user dengan data-data besar yang dimilikinya, hal ini dapat menimbulkan perangkat penyimpanan utama digital atau harddisc kapasitasnya penuh. Dengan demikian, dibutuhkan penyimpanan data berupa storage server yang digunakan sebagai cadangan penyimpanan data pada jaringan komputer. Penelitian ini merancang teknologi storage server dengan menggunakan perangkat lunak Apache Hadoop Distributed File System (HDFS) dan Java JDK yang diimplementasikan pada komputer Raspberry Pi sebagai komputer server, dengan menggunakan pembatasan kuota kapasitas penyimpanan. Rancangan storage server sebagai penyimpanan cadangan ini dapat digunakan bersama-sama namun dengan pembatan jumlah kapasitas dan directory. Pembatasan kuota kapasitas yang diberikan sebesar 10GB untuk masing-masing directory user. Hasil dari penelitian ini, storage server dapat digunakan secara bersama-sama oleh user, dimana masing-masing user dapat menyimpan data sebesar 10GB pada directory yang dimilikiny

Performance Engineering for Graduate Students:a View from Amsterdam

HPC relies on experts to design, implement, and tune (computational science) applications that can efficiently use current (super)computing systems. As such, we strongly believe we must educate our students to ensure their ability to drive these activities, together with the domain experts. To this end, in 2017, we have designed a performance engineering course that, inspired by several conference-like tutorials, covers the principles and practice of performance engineering: benchmarking, performance modeling, and performance improvement. In this paper, we describe the goals, learning objectives, and structure of the course, share students feedback and evaluation data, and discuss the lessons learned. After teaching the course seven times, our results show that the course is tough (as expected) but very well received, with high-scores and several students continuing on the path of performance engineering during and after their master studies.</p

Big Data-Oriented PaaS Architecture with Disk-as-a-Resource Capability and Container-Based Virtualization

This is a post-peer-review, pre-copyedit version of an article published in Journal of Grid Computing. The final authenticated version is available online at: https://doi.org/10.1007/s10723-018-9460-4[Abstract] With the increasing adoption of Big Data technologies as basic tools for the ongoing Digital Transformation, there is a high demand for data-intensive applications. In order to efficiently execute such applications, it is vital that cloud providers change the way hardware infrastructure resources are managed to improve their performance. However, the increasing use of virtualization technologies to achieve an efficient usage of infrastructure resources continuously widens the gap between applications and the underlying hardware, thus decreasing resource efficiency for the end user. Moreover, this scenario is especially troublesome for Big Data applications, as storage resources are one of the most heavily virtualized, thus imposing a significant overhead for large-scale data processing. This paper proposes a novel PaaS architecture specifically oriented for Big Data where the scheduler offers disks as resources alongside the more common CPU and memory resources, looking forward to provide a better storage solution for the user. Furthermore, virtualization overheads are reduced to the bare minimum by replacing heavy hypervisor-based technologies with operating-system-level virtualization based on light software containers. This architecture has been deployed on a Big Data infrastructure at the CESGA supercomputing center, used as a testbed to compare its performance with OpenStack, a popular private cloud platform. Results have shown significant performance improvements, reducing the execution time of representative Big Data workloads by up to 4.5×.Ministerio de Economía, Industria y Competitividad; TIN2016-75845-P, AEI/FEDER, EUMinisterio de Educación; FPU15/0338

Understanding the performance of interactive applications

Many if not most computer systems are used by human users. The performance of such interactive systems ultimately affects those users. Thus, when measuring, understanding, and improving system performance, it makes sense to consider the human user's perspective. Essentially, the performance of interactive applications is determined by the perceptible lag in handling user requests. So, when characterizing the runtime of an interactive application we need a new approach that focuses on the perceptible lags rather than on overall and general performance characteristics. Such a new characterization approach should enable a new way to profile and improve the performance of interactive applications. Imagine a way that would seek out these perceptible lags and then investigate the causes of these lags. Performance analysts could simply optimize responsible parts of the software, thus eliminating perceptible lag for interactive applications. Unfortunately, existing profiling approaches either incur significant overhead that makes them impractical for an interactive scenario, or they lack the ability to provide insight into the causes of long latencies. An effective approach for interactive applications has to fulfill several requirements such as an accurate view of the causes of performance problems and insignificant perturbation of the interactive application. We propose a new profiling approach that helps developers to understand and improve the perceptible performance of interactive applications and satisfies the above needs

A Survey of Enabling Technologies for Smart Communities

In 2016, the Japanese Government publicized an initiative and a call to action for the implementation of a Super Smart Society announced as Society 5.0. The stated goal of Society 5.0 is to meet the various needs of the members of society through the provisioning of goods and services to those who require them, when they are required and in the amount required, thus enabling the citizens to live an active and comfortable life. In spite of its genuine appeal, details of a feasible path to Society 5.0 are conspicuously missing. The first main goal of this survey is to suggest such an implementation path. Specifically, we define a Smart Community as a human-centric entity where technology is used to equip the citizenry with information and services that they can use to inform their decisions. The arbiter of this ecosystem of services is a Marketplace of Services that will reward services aligned with the wants and needs of the citizens, while discouraging the proliferation of those that are not. In the limit, the Smart Community we defined will morph into Society 5.0. At that point, the Marketplace of Services will become a platform for the co-creation of services by a close cooperation between the citizens and their government. The second objective and contribution of this survey paper is to review known technologies that, in our opinion, will play a significant role in the transition to Society 5.0. These technologies will be surveyed in chronological order, as newer technologies often extend old technologies while avoiding their limitations

Contribution au calcul sur GPU: considérations arithmétiques et architecturales

L’optimisation du calcul passe par une gestion conjointe du matériel et du logiciel. Cette règle se trouve renforcée lorsque l’on aborde le domaine des architectures multicoeurs où les paramètres à considérer sont plus nombreux que sur une architecture superscalaire classique. Ces architectures offrent une grande variété d’unité de calcul, de format de représentation, de hiérarchie mémoire et de mécanismes de transfert de donnée.Dans ce mémoire, nous décrivons quelques-uns de nos résultats obtenus entre 2004 et 2013 au sein de l'équipe DALI de l'Université de Perpignan relatifs à l'amélioration de l’efficacité du calcul dans sa globalité, c'est-à-dire dans la suite d’opérations décrite au niveau algorithmique et exécutées par les éléments architecturaux, en nous concentrant sur les processeurs graphiques.Nous commençons par une description du fonctionnement de ce type d'architecture, en nous attardant sur le calcul flottant. Nous présentons ensuite des implémentations efficaces d'opérateurs arithmétiques utilisant des représentations non-conventionnelles comme l'arithmétique multiprécision, par intervalle, floue ou logarithmique. Nous continuerons avec nos contributions relatives aux éléments architecturaux associés au calcul à travers la simulation fonctionnelle, les bancs de registres, la gestion des branchements ou les opérateurs matériels spécialisés. Enfin, nous terminerons avec une analyse du comportement du calcul sur les GPU relatif à la régularité, à la consommation électrique, à la fiabilisation des calculs ainsi qu'à laprédictibilité

Understanding and Improving the Performance of Read Operations Across the Storage Stack

We live in a data-driven era, large amounts of data are generated and collected every day. Storage systems are the backbone of this era, as they store and retrieve data. To cope with increasing data demands (e.g., diversity, scalability), storage systems are experiencing changes across the stack. As other computer systems, storage systems rely on layering and modularity, to allow rapid development. Unfortunately, this can hinder performance clarity and introduce degradations (e.g., tail latency), due to unexpected interactions between components of the stack. In this thesis, we first perform a study to understand the behavior across different layers of the storage stack. We focus on sequential read workloads, a common I/O pattern in distributed le systems (e.g., HDFS, GFS). We analyze the interaction between read workloads, local le systems (i.e., ext4), and storage media (i.e., SSDs). We perform the same experiment over different periods of time (e.g., le lifetime). We uncover 3 slowdowns, all of which occur in the lower layers. When combined, these slowdowns can degrade throughput by 30%. We find that increased parallelism on the local le system mitigates these slowdowns, showing the need for adaptability in storage stacks. Given the fact that performance instabilities can occur at any layer of the stack, it is important that upper-layer systems are able to react. We propose smart hedging, a novel technique to manage high-percentile (tail) latency variations in read operations. Smart hedging considers production challenges, such as massive scalability, heterogeneity, and ease of deployment and maintainability. Our technique establishes a dynamic threshold by tracking latencies on the client-side. If a read operation exceeds the threshold, a new hedged request is issued, in an exponential back-off manner. We implement our technique in HDFS and evaluate it on 70k servers in 3 datacenters. Our technique reduces average tail latency, without generating excessive system load