Convex and Network Flow Optimization for Structured Sparsity

We consider a class of learning problems regularized by a structured sparsity-inducing norm defined as the sum of l_2- or l_infinity-norms over groups of variables. Whereas much effort has been put in developing fast optimization techniques when the groups are disjoint or embedded in a hierarchy, we address here the case of general overlapping groups. To this end, we present two different strategies: On the one hand, we show that the proximal operator associated with a sum of l_infinity-norms can be computed exactly in polynomial time by solving a quadratic min-cost flow problem, allowing the use of accelerated proximal gradient methods. On the other hand, we use proximal splitting techniques, and address an equivalent formulation with non-overlapping groups, but in higher dimension and with additional constraints. We propose efficient and scalable algorithms exploiting these two strategies, which are significantly faster than alternative approaches. We illustrate these methods with several problems such as CUR matrix factorization, multi-task learning of tree-structured dictionaries, background subtraction in video sequences, image denoising with wavelets, and topographic dictionary learning of natural image patches.Comment: to appear in the Journal of Machine Learning Research (JMLR

Positron-Emission Tomography

We review positron-emission tomography (PET), which has inherent advantages that avoid the shortcomings of other nuclear medicine imaging methods. PET image reconstruction methods with origins in signal and image processing are discussed, including the potential problems of these methods. A summary of statistical image reconstruction methods, which can yield improved image quality, is also presented.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/85853/1/Fessler95.pd

Resource Management for Distributed Estimation via Sparsity-Promoting Regularization

Recent advances in wireless communications and electronics have enabled the development of low-cost, low-power, multifunctional sensor nodes that are small in size and communicate untethered in a sensor network. These sensor nodes can sense, measure, and gather information from the environment and, based on some local processing, they transmit the sensed data to a fusion center that is responsible for making the global inference. Sensor networks are often tasked to perform parameter estimation; example applications include battlefield surveillance, medical monitoring, and navigation. However, under limited resources, such as limited communication bandwidth and sensor battery power, it is important to design an energy-efficient estimation architecture. The goal of this thesis is to provide a fundamental understanding and characterization of the optimal tradeoffs between estimation accuracy and resource usage in sensor networks. In the thesis, two basic issues of resource management are studied, sensor selection/scheduling and sensor collaboration for distributed estimation, where the former refers to finding the best subset of sensors to activate for data acquisition in order to minimize the estimation error subject to a constraint on the number of activations, and the latter refers to seeking the optimal inter-sensor communication topology and energy allocation scheme for distributed estimation systems. Most research on resource management so far has been based on several key assumptions, a) independence of observation, b) strict resource constraints, and c) absence of inter-sensor communication, which lend analytical tractability to the problem but are often found lacking in practice. This thesis introduces novel techniques to relax these assumptions and provide new insights into addressing resource management problems. The thesis analyzes how noise correlation affects solutions of sensor selection problems, and proposes both a convex relaxation approach and a greedy algorithm to find these solutions. Compared to the existing sensor selection approaches that are limited to the case of uncorrelated noise or weakly correlated noise, the methodology proposed in this thesis is valid for any arbitrary noise correlation regime. Moreover, this thesis shows a correspondence between active sensors and the nonzero columns of an estimator gain matrix. Based on this association, a sparsity-promoting optimization framework is established, where the desire to reduce the number of selected sensors is characterized by a sparsity-promoting penalty term in the objective function. Instead of placing a hard constraint on sensor activations, the promotion of sparsity leads to trade-offs between estimation performance and the number of selected sensors. To account for the individual power constraint of each sensor, a novel sparsity-promoting penalty function is presented to avoid scenarios in which the same sensors are successively selected. For solving the proposed optimization problem, we employ the alternating direction method of multipliers (ADMM), which allows the optimization problem to be decomposed into subproblems that can be solved analytically to obtain exact solutions. The problem of sensor collaboration arises when inter-sensor communication is incorporated in sensor networks, where sensors are allowed to update their measurements by taking a linear combination of the measurements of those they interact with prior to transmission to a fusion center. In this thesis, a sparsity-aware optimization framework is presented for the joint design of optimal sensor collaboration and selection schemes, where the cost of sensor collaboration is associated with the number of nonzero entries of a collaboration matrix, and the cost of sensor selection is characterized by the number of nonzero rows of the collaboration matrix. It is shown that a) the presence of sensor collaboration smooths out the observation noise, thereby improving the quality of the signal and eventual estimation performance, and b) there exists a trade-off between sensor selection and sensor collaboration. This thesis further addresses the problem of sensor collaboration for the estimation of time-varying parameters in dynamic networks that involve, for example, time-varying observation gains and channel gains. Impact of parameter correlation and temporal dynamics of sensor networks on estimation performance is illustrated from both theoretical and practical points of view. Last but not least, optimal energy allocation and storage control polices are designed in sensor networks with energy-harvesting nodes. We show that the resulting optimization problem can be solved as a special nonconvex problem, where the only source of nonconvexity can be isolated to a constraint that contains the difference of convex functions. This specific problem structure enables the use of a convex-concave procedure to obtain a near-optimal solution

Computational Methods for Computer Vision : Minimal Solvers and Convex Relaxations

Robust fitting of geometric models is a core problem in computer vision. The most common approach is to use a hypothesize-and-test framework, such as RANSAC. In these frameworks the model is estimated from as few measurements as possible, which minimizes the risk of selecting corrupted measurements. These estimation problems are called minimal problems, and they can often be formulated as systems of polynomial equations. In this thesis we present new methods for building so-called minimal solvers or polynomial solvers, which are specialized code for solving such systems. On several minimal problems we improve on the state-of-the-art both with respect to numerical stability and execution time.In many computer vision problems low rank matrices naturally occur. The rank can serve as a measure of model complexity and typically a low rank is desired. Optimization problems containing rank penalties or constraints are in general difficult. Recently convex relaxations, such as the nuclear norm, have been used to make these problems tractable. In this thesis we present new convex relaxations for rank-based optimization which avoid drawbacks of previous approaches and provide tighter relaxations. We evaluate our methods on a number of real and synthetic datasets and show state-of-the-art results

Integration of process design and control: A review

There is a large variety of methods in literature for process design and control, which can be classified into two main categories. The methods in the first category have a sequential approach in which, the control system is designed, only after the details of process design are decided. However, when process design is fixed, there is little room left for improving the control performance. Recognizing the interactions between process design and control, the methods in the second category integrate some control aspects into process design. With the aim of providing an exploration map and identifying the potential areas of further contributions, this paper presents a thematic review of the methods for integration of process design and control. The evolution paths of these methods are described and the advantages and disadvantages of each method are explained. The paper concludes with suggestions for future research activities