A solution for secure use of Kibana and Elasticsearch in multi-user environment

Monitoring is indispensable to check status, activities, or resource usage of IT services. A combination of Kibana and Elasticsearch is used for monitoring in many places such as KEK, CC-IN2P3, CERN, and also non-HEP communities. Kibana provides a web interface for rich visualization, and Elasticsearch is a scalable distributed search engine. However, these tools do not support authentication and authorization features by default. In the case of single Kibana and Elasticsearch services shared among many users, any user who can access Kibana can retrieve other's information from Elasticsearch. In multi-user environment, in order to protect own data from others or share part of data among a group, fine-grained access control is necessary. The CERN cloud service group had provided cloud utilization dashboard to each user by Elasticsearch and Kibana. They had deployed a homemade Elasticsearch plugin to restrict data access based on a user authenticated by the CERN Single Sign On system. It enabled each user to have a separated Kibana dashboard for cloud usage, and the user could not access to other's one. Based on the solution, we propose an alternative one which enables user/group based Elasticsearch access control and Kibana objects separation. It is more flexible and can be applied to not only the cloud service but also the other various situations. We confirmed our solution works fine in CC-IN2P3. Moreover, a pre-production platform for CC-IN2P3 has been under construction. We will describe our solution for the secure use of Kibana and Elasticsearch including integration of Kerberos authentication, development of a Kibana plugin which allows Kibana objects to be separated based on user/group, and contribution to Search Guard which is an Elasticsearch plugin enabling user/group based access control. We will also describe the effect on performance from using Search Guard.Comment: International Symposium on Grids and Clouds 2017 (ISGC 2017

Experimental Performance Evaluation of Cloud-Based Analytics-as-a-Service

An increasing number of Analytics-as-a-Service solutions has recently seen the light, in the landscape of cloud-based services. These services allow flexible composition of compute and storage components, that create powerful data ingestion and processing pipelines. This work is a first attempt at an experimental evaluation of analytic application performance executed using a wide range of storage service configurations. We present an intuitive notion of data locality, that we use as a proxy to rank different service compositions in terms of expected performance. Through an empirical analysis, we dissect the performance achieved by analytic workloads and unveil problems due to the impedance mismatch that arise in some configurations. Our work paves the way to a better understanding of modern cloud-based analytic services and their performance, both for its end-users and their providers.Comment: Longer version of the paper in Submission at IEEE CLOUD'1

A template-based sub-optimal content distribution for D2D content sharing networks

We propose Templatized Elastic Assignment (TEA), a light-weight scheme for mobile cooperative caching networks. It consists of two components, (1) one to calculate a sub-optimal distribution of each situation and (2) finegrained ID management by base stations (BSs) to achieve the calculated distribution. The former is modeled from findings that the desirable distribution plotted in a semilog graph forms a downward straight line with which the slope and Yintercept epend on the bias of request and total cache capacity, respectively. The latter is inspired from the identifier (ID)-based scheme, which ties devices and content by a randomly associated ID. TEA achieved the calculated distribution with IDs by using the annotation from base stations (BSs), which is preliminarily calculated by the template in a fine-grained density of devices. Moreover, such fine-grained management secondarily standardizes the cached content among multiple densities and enables the reuse of the content in devices from other BSs. Evaluation results indicate that our scheme reduces (1) 8.3 times more traffic than LFU and achieves almost the same amount of traffic reduction as with the genetic algorithm, (2) 45 hours of computation into a few seconds, and (3) at most 70% of content replacement across multiple BSs