On Solving Selected Nonlinear Integer Programming Problems in Data Mining, Computational Biology, and Sustainability

This thesis consists of three essays concerning the use of optimization techniques to solve four problems in the fields of data mining, computational biology, and sustainable energy devices. To the best of our knowledge, the particular problems we discuss have not been previously addressed using optimization, which is a specific contribution of this dissertation. In particular, we analyze each of the problems to capture their underlying essence, subsequently demonstrating that each problem can be modeled as a nonlinear (mixed) integer program. We then discuss the design and implementation of solution techniques to locate optimal solutions to the aforementioned problems. Running throughout this dissertation is the theme of using mixed-integer programming techniques in conjunction with context-dependent algorithms to identify optimal and previously undiscovered underlying structure

Directly Coupled Observers for Quantum Harmonic Oscillators with Discounted Mean Square Cost Functionals and Penalized Back-action

This paper is concerned with quantum harmonic oscillators consisting of a quantum plant and a directly coupled coherent quantum observer. We employ discounted quadratic performance criteria in the form of exponentially weighted time averages of second-order moments of the system variables. A coherent quantum filtering (CQF) problem is formulated as the minimization of the discounted mean square of an estimation error, with which the dynamic variables of the observer approximate those of the plant. The cost functional also involves a quadratic penalty on the plant-observer coupling matrix in order to mitigate the back-action of the observer on the covariance dynamics of the plant. For the discounted mean square optimal CQF problem with penalized back-action, we establish first-order necessary conditions of optimality in the form of algebraic matrix equations. By using the Hamiltonian structure of the Heisenberg dynamics and related Lie-algebraic techniques, we represent this set of equations in a more explicit form in the case of equally dimensioned plant and observer.Comment: 11 pages, a brief version to be submitted to the IEEE 2016 Conference on Norbert Wiener in the 21st Century, 13-15 July, Melbourne, Australi

Maximum Persistency in Energy Minimization

We consider discrete pairwise energy minimization problem (weighted constraint satisfaction, max-sum labeling) and methods that identify a globally optimal partial assignment of variables. When finding a complete optimal assignment is intractable, determining optimal values for a part of variables is an interesting possibility. Existing methods are based on different sufficient conditions. We propose a new sufficient condition for partial optimality which is: (1) verifiable in polynomial time (2) invariant to reparametrization of the problem and permutation of labels and (3) includes many existing sufficient conditions as special cases. We pose the problem of finding the maximum optimal partial assignment identifiable by the new sufficient condition. A polynomial method is proposed which is guaranteed to assign same or larger part of variables than several existing approaches. The core of the method is a specially constructed linear program that identifies persistent assignments in an arbitrary multi-label setting.Comment: Extended technical report for the CVPR 2014 paper. Update: correction to the proof of characterization theore

Simultaneously Structured Models with Application to Sparse and Low-rank Matrices

The topic of recovery of a structured model given a small number of linear observations has been well-studied in recent years. Examples include recovering sparse or group-sparse vectors, low-rank matrices, and the sum of sparse and low-rank matrices, among others. In various applications in signal processing and machine learning, the model of interest is known to be structured in several ways at the same time, for example, a matrix that is simultaneously sparse and low-rank. Often norms that promote each individual structure are known, and allow for recovery using an order-wise optimal number of measurements (e.g., $\ell_1$ norm for sparsity, nuclear norm for matrix rank). Hence, it is reasonable to minimize a combination of such norms. We show that, surprisingly, if we use multi-objective optimization with these norms, then we can do no better, order-wise, than an algorithm that exploits only one of the present structures. This result suggests that to fully exploit the multiple structures, we need an entirely new convex relaxation, i.e. not one that is a function of the convex relaxations used for each structure. We then specialize our results to the case of sparse and low-rank matrices. We show that a nonconvex formulation of the problem can recover the model from very few measurements, which is on the order of the degrees of freedom of the matrix, whereas the convex problem obtained from a combination of the $\ell_1$ and nuclear norms requires many more measurements. This proves an order-wise gap between the performance of the convex and nonconvex recovery problems in this case. Our framework applies to arbitrary structure-inducing norms as well as to a wide range of measurement ensembles. This allows us to give performance bounds for problems such as sparse phase retrieval and low-rank tensor completion.Comment: 38 pages, 9 figure

Efficient graph cuts for unsupervised image segmentation using probabilistic sampling and SVD-based approximation

The application of graph theoretic methods to unsupervised image partitioning has been a very active field of research recently. For weighted graphs encoding the (dis)similarity structure of locally extracted image features, unsupervised segmentations of images into coherent structures can be computed in terms of extremal cuts of the underlying graphs. In this context, we focus on the normalized cut criterion and a related recent convex approach based on semidefinite programming. As both methods soon become computationally demanding with increasing graph size, an important question is how the computations can be accelerated. To this end, we study an SVD approximation method in this paper which has been introduced in a different clustering context. We apply this method, which is based on probabilistic sampling, to both segmentation approaches and compare it with the Nyström extension suggested for the normalized cut. Numerical results confirm that by means of the sampling-based SVD approximation technique, reliable segmentations can be computed with a fraction (less than 5%) of the original computational cost

Fewer is better: the cases of portfolio selection and of operational risk management

This thesis aims to show that in some applications the appropriate selection of a small number of available items can be beneficial with respect to the use of all available items. In particular, we focus on portfolio selection and on operational risk management and we use operations research techniques to identify the few important elements that are needed in both cases. In the first part of this work - based on an article published in Economics Bulletin [Cesarone et al (2016)], we show that, for several portfolio selection models, the best portfolio which uses only a limited number of assets has in-sample performance very close to that of an optimized portfolio which could include all assets, but generally obtains better out-of-sample performance. This is true for various performance measures, and it is often possible to identify a "golden range" of sizes where the best performances are obtained. These general empirical findings are consistent with theoretical results obtained by Kondor and Nagy (2007) under very restrictive assumptions. We also note that small portfolios are preferable for several practical reasons including monitoring, availability for small investors, and transaction costs. In the second part of the thesis, we develop an operational risk management framework for the assessment of the exposure of a company (with particular reference to a financial institution) to potential risk events arising from the launch of a new product. This framework is based on the Analytic Hierarchy Process and on the 80/20 rule which allows one to rank and to identify the most relevant risk events, respectively. By means of appropriate integer programming models we then address the problem of identifying the mitigation actions that secure the internal processes of a company with minimum cost. This corresponds to the primary goal of an operational risk manager: reducing the exposure to potential risk events. An alternative approach, when the budget is fixed, consists in selecting the subset of mitigation actions that provide the greatest reduction in operational risk exposure for that budget. A parametric analysis with respect to the budget level provides additional information for the management to take decisions about possible budget adjustments

Structured Sparsity: Discrete and Convex approaches

Compressive sensing (CS) exploits sparsity to recover sparse or compressible signals from dimensionality reducing, non-adaptive sensing mechanisms. Sparsity is also used to enhance interpretability in machine learning and statistics applications: While the ambient dimension is vast in modern data analysis problems, the relevant information therein typically resides in a much lower dimensional space. However, many solutions proposed nowadays do not leverage the true underlying structure. Recent results in CS extend the simple sparsity idea to more sophisticated {\em structured} sparsity models, which describe the interdependency between the nonzero components of a signal, allowing to increase the interpretability of the results and lead to better recovery performance. In order to better understand the impact of structured sparsity, in this chapter we analyze the connections between the discrete models and their convex relaxations, highlighting their relative advantages. We start with the general group sparse model and then elaborate on two important special cases: the dispersive and the hierarchical models. For each, we present the models in their discrete nature, discuss how to solve the ensuing discrete problems and then describe convex relaxations. We also consider more general structures as defined by set functions and present their convex proxies. Further, we discuss efficient optimization solutions for structured sparsity problems and illustrate structured sparsity in action via three applications.Comment: 30 pages, 18 figure

Defining block character

In this paper I propose a clear, efficient, and accurate method for determining if a block of contiguous buildings has an overall character. The work is needed because most contemporary design reviews presuppose the existence of visual character, but existing design principles are often too vague to make the required determination. Clarity is achieved by shifting from vague notions to a definite concept for block character: a design feature will be perceived as part of the overall character of that block if the frequency of the feature is greater than a critical threshold. An experiment suggested that the critical frequency was quite high: over 80%. A case history illustrates how the new concept of visual character could greatly increase the efficiency and accuracy of actual planning decisions.

Intentional Controlled Islanding in Wide Area Power Systems with Large Scale Renewable Power Generation to Prevent Blackout

Intentional controlled islanding is a solution to prevent blackouts following a large disturbance. This study focuses on determining island boundaries while maintaining the stability of formed islands and minimising load shedding. A new generator coherency identification framework based on the dynamic coupling of generators and Support Vector Clustering method is proposed to address this challenge. A Mixed Integer Linear Programming model is formulated to minimize power flow disruption and load shedding, and ensure the stability of islanding. The proposed algorithm was validated in 39-bus and 118-bus test systems