20,596 research outputs found
ALOJA: A benchmarking and predictive platform for big data performance analysis
The main goals of the ALOJA research project from BSC-MSR, are to explore and automate the characterization of cost-effectivenessof Big Data deployments. The development of the project over its first year, has resulted in a open source benchmarking platform, an online public repository of results with over 42,000 Hadoop job runs, and web-based analytic tools to gather insights about system's cost-performance1.
This article describes the evolution of the project's focus and research
lines from over a year of continuously benchmarking Hadoop under dif-
ferent configuration and deployments options, presents results, and dis
cusses the motivation both technical and market-based of such changes.
During this time, ALOJA's target has evolved from a previous low-level
profiling of Hadoop runtime, passing through extensive benchmarking
and evaluation of a large body of results via aggregation, to currently
leveraging Predictive Analytics (PA) techniques. Modeling benchmark
executions allow us to estimate the results of new or untested configu-
rations or hardware set-ups automatically, by learning techniques from
past observations saving in benchmarking time and costs.This work is partially supported the BSC-Microsoft Research Centre, the Span-
ish Ministry of Education (TIN2012-34557), the MINECO Severo Ochoa Research program (SEV-2011-0067) and the Generalitat de Catalunya (2014-SGR-1051).Peer ReviewedPostprint (author's final draft
Collaborative recommendations with content-based filters for cultural activities via a scalable event distribution platform
Nowadays, most people have limited leisure time and the offer of (cultural) activities to spend this time is enormous. Consequently, picking the most appropriate events becomes increasingly difficult for end-users. This complexity of choice reinforces the necessity of filtering systems that assist users in finding and selecting relevant events. Whereas traditional filtering tools enable e.g. the use of keyword-based or filtered searches, innovative recommender systems draw on user ratings, preferences, and metadata describing the events. Existing collaborative recommendation techniques, developed for suggesting web-shop products or audio-visual content, have difficulties with sparse rating data and can not cope at all with event-specific restrictions like availability, time, and location. Moreover, aggregating, enriching, and distributing these events are additional requisites for an optimal communication channel. In this paper, we propose a highly-scalable event recommendation platform which considers event-specific characteristics. Personal suggestions are generated by an advanced collaborative filtering algorithm, which is more robust on sparse data by extending user profiles with presumable future consumptions. The events, which are described using an RDF/OWL representation of the EventsML-G2 standard, are categorized and enriched via smart indexing and open linked data sets. This metadata model enables additional content-based filters, which consider event-specific characteristics, on the recommendation list. The integration of these different functionalities is realized by a scalable and extendable bus architecture. Finally, focus group conversations were organized with external experts, cultural mediators, and potential end-users to evaluate the event distribution platform and investigate the possible added value of recommendations for cultural participation
Fairs for e-commerce: the benefits of aggregating buyers and sellers
In recent years, many new and interesting models of successful online
business have been developed. Many of these are based on the competition
between users, such as online auctions, where the product price is not fixed
and tends to rise. Other models, including group-buying, are based on
cooperation between users, characterized by a dynamic price of the product that
tends to go down. There is not yet a business model in which both sellers and
buyers are grouped in order to negotiate on a specific product or service. The
present study investigates a new extension of the group-buying model, called
fair, which allows aggregation of demand and supply for price optimization, in
a cooperative manner. Additionally, our system also aggregates products and
destinations for shipping optimization. We introduced the following new
relevant input parameters in order to implement a double-side aggregation: (a)
price-quantity curves provided by the seller; (b) waiting time, that is, the
longer buyers wait, the greater discount they get; (c) payment time, which
determines if the buyer pays before, during or after receiving the product; (d)
the distance between the place where products are available and the place of
shipment, provided in advance by the buyer or dynamically suggested by the
system. To analyze the proposed model we implemented a system prototype and a
simulator that allow to study effects of changing some input parameters. We
analyzed the dynamic price model in fairs having one single seller and a
combination of selected sellers. The results are very encouraging and motivate
further investigation on this topic
AI Solutions for MDS: Artificial Intelligence Techniques for Misuse Detection and Localisation in Telecommunication Environments
This report considers the application of Articial Intelligence (AI) techniques to
the problem of misuse detection and misuse localisation within telecommunications
environments. A broad survey of techniques is provided, that covers inter alia
rule based systems, model-based systems, case based reasoning, pattern matching,
clustering and feature extraction, articial neural networks, genetic algorithms, arti
cial immune systems, agent based systems, data mining and a variety of hybrid
approaches. The report then considers the central issue of event correlation, that
is at the heart of many misuse detection and localisation systems. The notion of
being able to infer misuse by the correlation of individual temporally distributed
events within a multiple data stream environment is explored, and a range of techniques,
covering model based approaches, `programmed' AI and machine learning
paradigms. It is found that, in general, correlation is best achieved via rule based approaches,
but that these suffer from a number of drawbacks, such as the difculty of
developing and maintaining an appropriate knowledge base, and the lack of ability
to generalise from known misuses to new unseen misuses. Two distinct approaches
are evident. One attempts to encode knowledge of known misuses, typically within
rules, and use this to screen events. This approach cannot generally detect misuses
for which it has not been programmed, i.e. it is prone to issuing false negatives.
The other attempts to `learn' the features of event patterns that constitute normal
behaviour, and, by observing patterns that do not match expected behaviour, detect
when a misuse has occurred. This approach is prone to issuing false positives,
i.e. inferring misuse from innocent patterns of behaviour that the system was not
trained to recognise. Contemporary approaches are seen to favour hybridisation,
often combining detection or localisation mechanisms for both abnormal and normal
behaviour, the former to capture known cases of misuse, the latter to capture
unknown cases. In some systems, these mechanisms even work together to update
each other to increase detection rates and lower false positive rates. It is concluded
that hybridisation offers the most promising future direction, but that a rule or state
based component is likely to remain, being the most natural approach to the correlation
of complex events. The challenge, then, is to mitigate the weaknesses of
canonical programmed systems such that learning, generalisation and adaptation
are more readily facilitated
HAPPY: Hybrid Address-based Page Policy in DRAMs
Memory controllers have used static page closure policies to decide whether a
row should be left open, open-page policy, or closed immediately, close-page
policy, after the row has been accessed. The appropriate choice for a
particular access can reduce the average memory latency. However, since
application access patterns change at run time, static page policies cannot
guarantee to deliver optimum execution time. Hybrid page policies have been
investigated as a means of covering these dynamic scenarios and are now
implemented in state-of-the-art processors. Hybrid page policies switch between
open-page and close-page policies while the application is running, by
monitoring the access pattern of row hits/conflicts and predicting future
behavior. Unfortunately, as the size of DRAM memory increases, fine-grain
tracking and analysis of memory access patterns does not remain practical. We
propose a compact memory address-based encoding technique which can improve or
maintain the performance of DRAMs page closure predictors while reducing the
hardware overhead in comparison with state-of-the-art techniques. As a case
study, we integrate our technique, HAPPY, with a state-of-the-art monitor, the
Intel-adaptive open-page policy predictor employed by the Intel Xeon X5650, and
a traditional Hybrid page policy. We evaluate them across 70 memory intensive
workload mixes consisting of single-thread and multi-thread applications. The
experimental results show that using the HAPPY encoding applied to the
Intel-adaptive page closure policy can reduce the hardware overhead by 5X for
the evaluated 64 GB memory (up to 40X for a 512 GB memory) while maintaining
the prediction accuracy
PaPaS: A Portable, Lightweight, and Generic Framework for Parallel Parameter Studies
The current landscape of scientific research is widely based on modeling and
simulation, typically with complexity in the simulation's flow of execution and
parameterization properties. Execution flows are not necessarily
straightforward since they may need multiple processing tasks and iterations.
Furthermore, parameter and performance studies are common approaches used to
characterize a simulation, often requiring traversal of a large parameter
space. High-performance computers offer practical resources at the expense of
users handling the setup, submission, and management of jobs. This work
presents the design of PaPaS, a portable, lightweight, and generic workflow
framework for conducting parallel parameter and performance studies. Workflows
are defined using parameter files based on keyword-value pairs syntax, thus
removing from the user the overhead of creating complex scripts to manage the
workflow. A parameter set consists of any combination of environment variables,
files, partial file contents, and command line arguments. PaPaS is being
developed in Python 3 with support for distributed parallelization using SSH,
batch systems, and C++ MPI. The PaPaS framework will run as user processes, and
can be used in single/multi-node and multi-tenant computing systems. An example
simulation using the BehaviorSpace tool from NetLogo and a matrix multiply
using OpenMP are presented as parameter and performance studies, respectively.
The results demonstrate that the PaPaS framework offers a simple method for
defining and managing parameter studies, while increasing resource utilization.Comment: 8 pages, 6 figures, PEARC '18: Practice and Experience in Advanced
Research Computing, July 22--26, 2018, Pittsburgh, PA, US
- …