3,038 research outputs found
Communication-Efficient Algorithms For Distributed Optimization
This thesis is concerned with the design of distributed algorithms for
solving optimization problems. We consider networks where each node has
exclusive access to a cost function, and design algorithms that make all nodes
cooperate to find the minimum of the sum of all the cost functions. Several
problems in signal processing, control, and machine learning can be posed as
such optimization problems. Given that communication is often the most
energy-consuming operation in networks, it is important to design
communication-efficient algorithms. The main contributions of this thesis are a
classification scheme for distributed optimization and a set of corresponding
communication-efficient algorithms.
The class of optimization problems we consider is quite general, since each
function may depend on arbitrary components of the optimization variable, and
not necessarily on all of them. In doing so, we go beyond the common assumption
in distributed optimization and create additional structure that can be used to
reduce the number of communications. This structure is captured by our
classification scheme, which identifies easier instances of the problem, for
example the standard distributed optimization problem, where all functions
depend on all the components of the variable.
In our algorithms, no central node coordinates the network, all the
communications occur between neighboring nodes, and the data associated with
each node is processed locally. We show several applications including average
consensus, support vector machines, network flows, and several distributed
scenarios for compressed sensing. We also propose a new framework for
distributed model predictive control. Through extensive numerical experiments,
we show that our algorithms outperform prior distributed algorithms in terms of
communication-efficiency, even some that were specifically designed for a
particular application.Comment: Thesis defended on October 10, 2013. Dual PhD degree from Carnegie
Mellon University, PA, and Instituto Superior T\'ecnico, Lisbon, Portuga
Multi-task CNN Model for Attribute Prediction
This paper proposes a joint multi-task learning algorithm to better predict
attributes in images using deep convolutional neural networks (CNN). We
consider learning binary semantic attributes through a multi-task CNN model,
where each CNN will predict one binary attribute. The multi-task learning
allows CNN models to simultaneously share visual knowledge among different
attribute categories. Each CNN will generate attribute-specific feature
representations, and then we apply multi-task learning on the features to
predict their attributes. In our multi-task framework, we propose a method to
decompose the overall model's parameters into a latent task matrix and
combination matrix. Furthermore, under-sampled classifiers can leverage shared
statistics from other classifiers to improve their performance. Natural
grouping of attributes is applied such that attributes in the same group are
encouraged to share more knowledge. Meanwhile, attributes in different groups
will generally compete with each other, and consequently share less knowledge.
We show the effectiveness of our method on two popular attribute datasets.Comment: 11 pages, 3 figures, ieee transaction pape
- …