15,713 research outputs found
Streaming, Memory Limited Algorithms for Community Detection
In this paper, we consider sparse networks consisting of a finite number of
non-overlapping communities, i.e. disjoint clusters, so that there is higher
density within clusters than across clusters. Both the intra- and inter-cluster
edge densities vanish when the size of the graph grows large, making the
cluster reconstruction problem nosier and hence difficult to solve. We are
interested in scenarios where the network size is very large, so that the
adjacency matrix of the graph is hard to manipulate and store. The data stream
model in which columns of the adjacency matrix are revealed sequentially
constitutes a natural framework in this setting. For this model, we develop two
novel clustering algorithms that extract the clusters asymptotically
accurately. The first algorithm is {\it offline}, as it needs to store and keep
the assignments of nodes to clusters, and requires a memory that scales
linearly with the network size. The second algorithm is {\it online}, as it may
classify a node when the corresponding column is revealed and then discard this
information. This algorithm requires a memory growing sub-linearly with the
network size. To construct these efficient streaming memory-limited clustering
algorithms, we first address the problem of clustering with partial
information, where only a small proportion of the columns of the adjacency
matrix is observed and develop, for this setting, a new spectral algorithm
which is of independent interest.Comment: NIPS 201
Stream Learning in Energy IoT Systems: A Case Study in Combined Cycle Power Plants
The prediction of electrical power produced in combined cycle power plants is a key challenge in the electrical power and energy systems field. This power production can vary depending on environmental variables, such as temperature, pressure, and humidity. Thus, the business problem is how to predict the power production as a function of these environmental conditions, in order to maximize the profit. The research community has solved this problem by applying Machine Learning techniques, and has managed to reduce the computational and time costs in comparison with the traditional thermodynamical analysis. Until now, this challenge has been tackled from a batch learning perspective, in which data is assumed to be at rest, and where models do not continuously integrate new information into already constructed models. We present an approach closer to the Big Data and Internet of Things paradigms, in which data are continuously arriving and where models learn incrementally, achieving significant enhancements in terms of data processing (time, memory and computational costs), and obtaining competitive performances. This work compares and examines the hourly electrical power prediction of several streaming regressors, and discusses about the best technique in terms of time processing and predictive performance to be applied on this streaming scenario.This work has been partially supported by the EU project iDev40. This project has received funding
from the ECSEL Joint Undertaking (JU) under grant agreement No 783163. The JU receives support from the
European Union’s Horizon 2020 research and innovation programme and Austria, Germany, Belgium, Italy,
Spain, Romania. It has also been supported by the Basque Government (Spain) through the project VIRTUAL
(KK-2018/00096), and by Ministerio de EconomĂa y Competitividad of Spain (Grant Ref. TIN2017-85887-C2-2-P)
Fully Dynamic Algorithm for Top- Densest Subgraphs
Given a large graph, the densest-subgraph problem asks to find a subgraph
with maximum average degree. When considering the top- version of this
problem, a na\"ive solution is to iteratively find the densest subgraph and
remove it in each iteration. However, such a solution is impractical due to
high processing cost. The problem is further complicated when dealing with
dynamic graphs, since adding or removing an edge requires re-running the
algorithm. In this paper, we study the top- densest-subgraph problem in the
sliding-window model and propose an efficient fully-dynamic algorithm. The
input of our algorithm consists of an edge stream, and the goal is to find the
node-disjoint subgraphs that maximize the sum of their densities. In contrast
to existing state-of-the-art solutions that require iterating over the entire
graph upon any update, our algorithm profits from the observation that updates
only affect a limited region of the graph. Therefore, the top- densest
subgraphs are maintained by only applying local updates. We provide a
theoretical analysis of the proposed algorithm and show empirically that the
algorithm often generates denser subgraphs than state-of-the-art competitors.
Experiments show an improvement in efficiency of up to five orders of magnitude
compared to state-of-the-art solutions.Comment: 10 pages, 8 figures, accepted at CIKM 201
- …