1,963 research outputs found
The H.E.S.S. central data acquisition system
The High Energy Stereoscopic System (H.E.S.S.) is a system of Imaging
Atmospheric Cherenkov Telescopes (IACTs) located in the Khomas Highland in
Namibia. It measures cosmic gamma rays of very high energies (VHE; >100 GeV)
using the Earth's atmosphere as a calorimeter. The H.E.S.S. Array entered Phase
II in September 2012 with the inauguration of a fifth telescope that is larger
and more complex than the other four. This paper will give an overview of the
current H.E.S.S. central data acquisition (DAQ) system with particular emphasis
on the upgrades made to integrate the fifth telescope into the array. At first,
the various requirements for the central DAQ are discussed then the general
design principles employed to fulfil these requirements are described. Finally,
the performance, stability and reliability of the H.E.S.S. central DAQ are
presented. One of the major accomplishments is that less than 0.8% of
observation time has been lost due to central DAQ problems since 2009.Comment: 17 pages, 8 figures, published in Astroparticle Physic
CloudTPS: Scalable Transactions for Web Applications in the Cloud
NoSQL Cloud data services provide scalability and high availability properties for web applications but at the same time they sacrifice data consistency. However, many applications cannot afford any data inconsistency. CloudTPS is a scalable transaction manager to allow cloud database services to execute the ACID transactions of web applications, even in the presence of server failures and network partitions. We implement this approach on top of the two main families of scalable data layers: Bigtable and SimpleDB. Performance evaluation on top of HBase (an open-source version of Bigtable) in our local cluster and Amazon SimpleDB in the Amazon cloud shows that our system scales linearly at least up to 40 nodes in our local cluster and 80 nodes in the Amazon cloud
LenticularFS: Scalable filesystem for the cloud
The Hadoop platform is the most common solution to handle the explosion of big-data that both companies and research institutions are facing. In order to store such data, the Hadoop platform provides HDFS, a scalable distributed filesystem, which runs on commodity hardware and enables linear scalability by adding new storage nodes. While storage capacity of the system can be increased by adding new storage nodes, the component that handles metadata for the filesystem, the namenode, is a single point of failure and cannot easily replaced or linearly scaled. The Hops projects provides an alternative implementation of the namenode, which increases performance and scalability by storing metadata on an external distributed NewSQL database called MySQL Cluster. With the new architecture, the system is much more scalable and can transparently manage the failover of namenodes, which are now stateless components. HopsFS is, however, still limited to running within a single datacenter, which can cause severe outages in case the entire datacenter becomes unavailable. Cloud native storage systems, such as Amazon’s Simple Storage Service (S3), solve this problem by replicating data across different, geographically distant datacenters, so that the failure of any given zone does not cause data unavailability. The objective of this thesis is to enable HopsFS to work across geographical regions while, as far as possible, maintaining the semantics of a POSIX-style hierarchical filesystem. We leverage asynchronous replication functionality provided by MySQL Cluster to obtain replication of metadata across geographical regions and we present a detailed analysis on how to maintain the consistency properties of HDFS in such an environment. Furthermore, we analyze the issue of split brain scenarios and propose a way for namenodes to detect this condition and continue operating correctly. Finally, we discuss the changes to the codebase which are required to implement the proposed plan
Database Principles and Technologies – Based on Huawei GaussDB
This open access book contains eight chapters that deal with database technologies, including the development history of database, database fundamentals, introduction to SQL syntax, classification of SQL syntax, database security fundamentals, database development environment, database design fundamentals, and the application of Huawei’s cloud database product GaussDB database. This book can be used as a textbook for database courses in colleges and universities, and is also suitable as a reference book for the HCIA-GaussDB V1.5 certification examination. The Huawei GaussDB (for MySQL) used in the book is a Huawei cloud-based high-performance, highly applicable relational database that fully supports the syntax and functionality of the open source database MySQL. All the experiments in this book can be run on this database platform. As the world’s leading provider of ICT (information and communication technology) infrastructure and smart terminals, Huawei’s products range from digital data communication, cyber security, wireless technology, data storage, cloud computing, and smart computing to artificial intelligence
Benchmarking of RDBMS and NoSQL performance on unstructured data
New requirements are arising in the database field. Big data has been soaring.
The amount of data is ever increasing and becoming more and more varied. Traditional
relational database management systems have been a dominant force in the database field
but due to the massive growth of unstructured and multiform data, firms are now turning to
architectures that have scaleout
capabilities using open source software, commodity
servers, cloud computing and services like Database as a Service. Due to this, relational
databases ought to adopt and meet these new data requirements with easier and faster data
processing capabilities and also provide multiple analytical tools that have the possibility of
displaying analytics instantly. This study aims to benchmark the performance of relational
systems and NoSQL systems on unstructured data.Novos requisitos estão surgindo na área das bases de dados. “Big data” permitiu avanços
consideráveis em vários setores.
O volume de dados tem aumentado e tornase
cada vez mais variado. Os sistemas
tradicionais de gestão de base de dados relacionais têm sido uma força dominante na área,
mas devido ao crescimento massivo de dados nĂŁo estruturados e multiformes, as empresas
agora recorrem a arquiteturas que possuem recursos escaláveis usando software livre,
servidores, computação em nuvem e serviços, tais como “base de dados como um serviço”.
Nesse sentido, as bases de dados relacionais devem considerar e adotar novos requisitos
de dados com maior agilidade no seu processamento e também fornecer múltiplas
ferramentas analĂticas com a possibilidade de mostrar análises em tempo real. Este estudo
tem como objetivo avaliar o desempenho de sistemas relacionais e sistemas NoSQL em
dados nĂŁo estruturados
- …