72,200 research outputs found
DeepOBS: A Deep Learning Optimizer Benchmark Suite
Because the choice and tuning of the optimizer affects the speed, and
ultimately the performance of deep learning, there is significant past and
recent research in this area. Yet, perhaps surprisingly, there is no generally
agreed-upon protocol for the quantitative and reproducible evaluation of
optimization strategies for deep learning. We suggest routines and benchmarks
for stochastic optimization, with special focus on the unique aspects of deep
learning, such as stochasticity, tunability and generalization. As the primary
contribution, we present DeepOBS, a Python package of deep learning
optimization benchmarks. The package addresses key challenges in the
quantitative assessment of stochastic optimizers, and automates most steps of
benchmarking. The library includes a wide and extensible set of ready-to-use
realistic optimization problems, such as training Residual Networks for image
classification on ImageNet or character-level language prediction models, as
well as popular classics like MNIST and CIFAR-10. The package also provides
realistic baseline results for the most popular optimizers on these test
problems, ensuring a fair comparison to the competition when benchmarking new
optimizers, and without having to run costly experiments. It comes with output
back-ends that directly produce LaTeX code for inclusion in academic
publications. It supports TensorFlow and is available open source.Comment: Accepted at ICLR 2019. 9 pages, 3 figures, 2 table
On Resource Allocation in Fading Multiple Access Channels - An Efficient Approximate Projection Approach
We consider the problem of rate and power allocation in a multiple-access
channel. Our objective is to obtain rate and power allocation policies that
maximize a general concave utility function of average transmission rates on
the information theoretic capacity region of the multiple-access channel. Our
policies does not require queue-length information. We consider several
different scenarios. First, we address the utility maximization problem in a
nonfading channel to obtain the optimal operating rates, and present an
iterative gradient projection algorithm that uses approximate projection. By
exploiting the polymatroid structure of the capacity region, we show that the
approximate projection can be implemented in time polynomial in the number of
users. Second, we consider resource allocation in a fading channel. Optimal
rate and power allocation policies are presented for the case that power
control is possible and channel statistics are available. For the case that
transmission power is fixed and channel statistics are unknown, we propose a
greedy rate allocation policy and provide bounds on the performance difference
of this policy and the optimal policy in terms of channel variations and
structure of the utility function. We present numerical results that
demonstrate superior convergence rate performance for the greedy policy
compared to queue-length based policies. In order to reduce the computational
complexity of the greedy policy, we present approximate rate allocation
policies which track the greedy policy within a certain neighborhood that is
characterized in terms of the speed of fading.Comment: 32 pages, Submitted to IEEE Trans. on Information Theor
Power System Parameters Forecasting Using Hilbert-Huang Transform and Machine Learning
A novel hybrid data-driven approach is developed for forecasting power system
parameters with the goal of increasing the efficiency of short-term forecasting
studies for non-stationary time-series. The proposed approach is based on mode
decomposition and a feature analysis of initial retrospective data using the
Hilbert-Huang transform and machine learning algorithms. The random forests and
gradient boosting trees learning techniques were examined. The decision tree
techniques were used to rank the importance of variables employed in the
forecasting models. The Mean Decrease Gini index is employed as an impurity
function. The resulting hybrid forecasting models employ the radial basis
function neural network and support vector regression. Apart from introduction
and references the paper is organized as follows. The section 2 presents the
background and the review of several approaches for short-term forecasting of
power system parameters. In the third section a hybrid machine learning-based
algorithm using Hilbert-Huang transform is developed for short-term forecasting
of power system parameters. Fourth section describes the decision tree learning
algorithms used for the issue of variables importance. Finally in section six
the experimental results in the following electric power problems are
presented: active power flow forecasting, electricity price forecasting and for
the wind speed and direction forecasting
- …