870 research outputs found
Conditional Restricted Boltzmann Machines for Structured Output Prediction
Conditional Restricted Boltzmann Machines (CRBMs) are rich probabilistic
models that have recently been applied to a wide range of problems, including
collaborative filtering, classification, and modeling motion capture data.
While much progress has been made in training non-conditional RBMs, these
algorithms are not applicable to conditional models and there has been almost
no work on training and generating predictions from conditional RBMs for
structured output problems. We first argue that standard Contrastive
Divergence-based learning may not be suitable for training CRBMs. We then
identify two distinct types of structured output prediction problems and
propose an improved learning algorithm for each. The first problem type is one
where the output space has arbitrary structure but the set of likely output
configurations is relatively small, such as in multi-label classification. The
second problem is one where the output space is arbitrarily structured but
where the output space variability is much greater, such as in image denoising
or pixel labeling. We show that the new learning algorithms can work much
better than Contrastive Divergence on both types of problems
Weighted Contrastive Divergence
Learning algorithms for energy based Boltzmann architectures that rely on
gradient descent are in general computationally prohibitive, typically due to
the exponential number of terms involved in computing the partition function.
In this way one has to resort to approximation schemes for the evaluation of
the gradient. This is the case of Restricted Boltzmann Machines (RBM) and its
learning algorithm Contrastive Divergence (CD). It is well-known that CD has a
number of shortcomings, and its approximation to the gradient has several
drawbacks. Overcoming these defects has been the basis of much research and new
algorithms have been devised, such as persistent CD. In this manuscript we
propose a new algorithm that we call Weighted CD (WCD), built from small
modifications of the negative phase in standard CD. However small these
modifications may be, experimental work reported in this paper suggest that WCD
provides a significant improvement over standard CD and persistent CD at a
small additional computational cost
Energy Disaggregation for Real-Time Building Flexibility Detection
Energy is a limited resource which has to be managed wisely, taking into
account both supply-demand matching and capacity constraints in the
distribution grid. One aspect of the smart energy management at the building
level is given by the problem of real-time detection of flexible demand
available. In this paper we propose the use of energy disaggregation techniques
to perform this task. Firstly, we investigate the use of existing
classification methods to perform energy disaggregation. A comparison is
performed between four classifiers, namely Naive Bayes, k-Nearest Neighbors,
Support Vector Machine and AdaBoost. Secondly, we propose the use of Restricted
Boltzmann Machine to automatically perform feature extraction. The extracted
features are then used as inputs to the four classifiers and consequently shown
to improve their accuracy. The efficiency of our approach is demonstrated on a
real database consisting of detailed appliance-level measurements with high
temporal resolution, which has been used for energy disaggregation in previous
studies, namely the REDD. The results show robustness and good generalization
capabilities to newly presented buildings with at least 96% accuracy.Comment: To appear in IEEE PES General Meeting, 2016, Boston, US
- …