A scalable monitoring for the CMS Filter Farm based on elasticsearch

A flexible monitoring system has been designed for the CMS File-based Filter Farm making use of modern data mining and analytics components. All the metadata and monitoring information concerning data flow and execution of the HLT are generated locally in the form of small documents using the JSON encoding. These documents are indexed into a hierarchy of elasticsearch (es) clusters along with process and system log information. Elasticsearch is a search server based on Apache Lucene. It provides a distributed, multitenant-capable search and aggregation engine. Since es is schema-free, any new information can be added seamlessly and the unstructured information can be queried in non-predetermined ways. The leaf es clusters consist of the very same nodes that form the Filter Farm thus providing natural horizontal scaling. A separate central" es cluster is used to collect and index aggregated information. The fine-grained information, all the way to individual processes, remains available in the leaf clusters. The central es cluster provides quasi-real-time high-level monitoring information to any kind of client. Historical data can be retrieved to analyse past problems or correlate them with external information. We discuss the design and performance of this system in the context of the CMS DAQ commissioning for LHC Run 2

Example of IoT platform usage for wireless video surveillance with support of NoSQL and cloud systems

Today there is an increasing need for implementation of area security systems, especially in sense of monitoring areas of interest. Many of the solutions related to wireless security cameras that are available on the market are mainly limited with their predefined functionalities. Furthermore, these sets of functionalities largely affect price levels. Thanks to an increasing development and availability of open-source hardware and DIY (Do It Youself) electronics applicable in the field of Internet of Things (IoT), as well as new methods in data storage, such as NoSQL, new opportunities for creation of custom systems for video monitoring and storing video data are opened. Through this work it is presented the solution for wireless security cameras, based on the IoT enabled open-source hardware and MongoDB database as the storage system. Also, in order to achieve replication of created content, possibilities of storing this content on the cloud storage system are explored. Established solution can be used on a\ud daily basis, both in the private and business environments. Also, in this paper are presented technologies used for system development. The solution can be used as a starting point for further development of the systems for areal monitoring and content of interest creation using the emerging technologies

Cassandra File System Over Hadoop Distributed File System

Cassandra is an open source distributed database management system is designed to handle large amounts of data across many commodity servers, provides a high availability with no single point of failure. Cassandra will be offering the robust support for clusters spanning multiple data centers with asynchronous masterless replica which allow low latency operations for all the clients. N oSQL data stores target the unstructured data, which nature has dynamic and a key focus area for "Big Data" research. New generation data can prove costly and also unpractical to administer with databases SQL, due to lack of structure, high scalability and needs for the elasticity. N oSQL data stores such as MongoDB and Cassandra provide a desirable platform for fast and efficient for data queries. The Hadoop Distributed File System is one of many different components and projects contained within the community Hadoop ecosystem. The Apache Hadoop project defines Had oop - DFS as "the primary storage system which is used by Hadoop applications" that enables "reliable, extremely rapid computations". This paper was providing high - level overview of how Hadoop - styled analytics (MapReduce, Pig, Mahout and Hive) can be run on data contained in Apache Cassandra wit hout the need for Hadoop - DFS

Transforming Home Appliances into IoT Devices

Home appliances were, until not very recently, isolated devices uncapable of communicate with others. With the recent technological advances is it possible nowadays to connect all these devices to the internet enabling them with the capacity to communicate between them and also the possibility to store and analyse the data generated making them more efficient. Due to these advancements, terms like Internet of Things and Cloud Computing surfaced. These terms represent architectures that allow communications and interactions between devices. The potential of these technologies is substantial for the improvement of productivity and eco-nomic impact. These advancements allowed home appliances to generate an enormous amount of data, and as repercussion, the need for structures able to receive, storage and process the data emerged. The present document contains a proposition of an architecture that allows to collect and send home appliances’ data to the Cloud, a way to store and make them available to other applications and a platform that allows the data’s visualisation and analysis. The proposed architecture was tested with a refrigerator offered by Electrolux inserted in the ProSEcO project

Using the MEAN Stack to implement a RESTful service for an Internet of Things Application

This paper examines the components of the MEAN development stack (MongoDb, Express.js, Angular.js, & Node.js), and demonstrate their benefits and appropriateness to be used in implementing RESTful web-service APIs for Inter- net of Things (IoT) appliances. In particular, we show an end- to-end example of this stack and discuss in detail the various components required. The paper also describes an approach to establishing a secure mechanism for communicating with IoT devices, using pull-communications

SAP HANA Database: Data Management for Modern Business Applications

The SAP HANA database is positioned as the core of the SAP HANA Appliance to support complex business analytical processes in combination with transactionally consistent operational workloads. Within this paper, we outline the basic characteristics of the SAP HANA database, emphasizing the distinctive features that differentiate the SAP HANA database from other classical relational database management systems. On the technical side, the SAP HANA database consists of multiple data processing engines with a distributed query processing environment to provide the full spectrum of data processing -- from classical relational data supporting both row- and column-oriented physical representations in a hybrid engine, to graph and text processing for semi- and unstructured data management within the same system. From a more application-oriented perspective, we outline the specific support provided by the SAP HANA database of multiple domain-specific languages with a built-in set of natively implemented business functions. SQL -- as the lingua franca for relational database systems -- can no longer be considered to meet all requirements of modern applications, which demand the tight interaction with the data management layer. Therefore, the SAP HANA database permits the exchange of application semantics with the underlying data management platform that can be exploited to increase query expressiveness and to reduce the number of individual application-to-database round trips