An almost cyclic 2-coordinate descent method for singly linearly constrained problems

A block decomposition method is proposed for minimizing a (possibly non-convex) continuously differentiable function subject to one linear equality constraint and simple bounds on the variables. The proposed method iteratively selects a pair of coordinates according to an almost cyclic strategy that does not use first-order information, allowing us not to compute the whole gradient of the objective function during the algorithm. Using first-order search directions to update each pair of coordinates, global convergence to stationary points is established for different choices of the stepsize under an appropriate assumption on the level set. In particular, both inexact and exact line search strategies are analyzed. Further, linear convergence rate is proved under standard additional assumptions. Numerical results are finally provided to show the effectiveness of the proposed method.Comment: Computational Optimization and Application

A decomposition method for lasso problems with zero-sum constraint

In this paper, we consider lasso problems with zero-sum constraint, commonly required for the analysis of compositional data in high-dimensional spaces. A novel algorithm is proposed to solve these problems, combining a tailored active-set technique, to identify the zero variables in the optimal solution, with a 2-coordinate descent scheme. At every iteration, the algorithm chooses between two different strategies: the first one requires to compute the whole gradient of the smooth term of the objective function and is more accurate in the active-set estimate, while the second one only uses partial derivatives and is computationally more efficient. Global convergence to optimal solutions is proved and numerical results are provided on synthetic and real datasets, showing the effectiveness of the proposed method. The software is publicly available

Active-set identification with complexity guarantees of an almost cyclic 2-coordinate descent method with Armijo line search

In this paper, it is established finite active-set identification of an almost cyclic 2-coordinate descent method for problems with one linear coupling constraint and simple bounds. First, general active-set identification results are stated for non-convex objective functions. Then, under convexity and a quadratic growth condition (satisfied by any strongly convex function), complexity results on the number of iterations required to identify the active set are given. In our analysis, a simple Armijo line search is used to compute the stepsize, thus not requiring exact minimizations or additional information

A Cyclic Coordinate Descent Method for Convex Optimization on Polytopes

Coordinate descent algorithms are popular for huge-scale optimization problems due to their low cost per-iteration. Coordinate descent methods apply to problems where the constraint set is separable across coordinates. In this paper, we propose a new variant of the cyclic coordinate descent method that can handle polyhedral constraints provided that the polyhedral set does not have too many extreme points such as L1-ball and the standard simplex. Loosely speaking, our proposed algorithm PolyCD, can be viewed as a hybrid of cyclic coordinate descent and the Frank-Wolfe algorithms. We prove that PolyCD has a O(1/k) convergence rate for smooth convex objectives. Inspired by the away-step variant of Frank-Wolfe, we propose PolyCDwA, a variant of PolyCD with away steps which has a linear convergence rate when the loss function is smooth and strongly convex. Empirical studies demonstrate that PolyCDwA achieves strong computational performance for large-scale benchmark problems including L1-constrained linear regression, L1-constrained logistic regression and kernel density estimation

Facial feature localization using highly flexible yet sufficiently strict shape models

textAccurate and efficient localization of facial features is a crucial first step in many face-related computer vision tasks. Some of these tasks include, but not limited to: identity recognition, expression recognition, and head-pose estimation. Most effort in the field has been exerted towards developing better ways of modeling prior appearance knowledge and image observations. Modeling prior shape knowledge, on the other hand, has not been explored as much. In this dissertation I primarily focus on the limitations of the existing methods in terms of modeling the prior shape knowledge. I first introduce a new pose-constrained shape model. I describe my shape model as being "highly flexible yet sufficiently strict". Existing pose-constrained shape models are either too strict, and have questionable generalization power, or they are too loose, and have questionable localization accuracies. My model tries to find a good middle-ground by learning which shape constraints are more "informative" and should be kept, and which ones are not-so-important and may be omitted. I build my pose-constrained facial feature localization approach on this new shape model using a probabilistic graphical model framework. Within this framework, observed and unobserved variables are defined as the local image observations, and the feature locations, respectively. Feature localization, or "probabilistic inference", is then achieved by nonparametric belief propagation. I show that this approach outperforms other popular pose-constrained methods through qualitative and quantitative experiments. Next, I expand my pose-constrained localization approach to unconstrained setting using a multi-model strategy. While doing so, once again I identify and address the two key limitations of existing multi-model methods: 1) semantically and manually defining the models or "guiding" their generation, and 2) not having efficient and effective model selection strategies. First, I introduce an approach based on unsupervised clustering where the models are automatically learned from training data. Then, I complement this approach with an efficient and effective model selection strategy, which is based on a multi-class naive Bayesian classifier. This way, my method can have many more models, each with a higher level of expressive power, and consequently, provides a more effective partitioning of the face image space. This approach is validated through extensive experiments and comparisons with state-of-the-art methods on state-of-the-art datasets. In the last part of this dissertation I discuss a particular application of the previously introduced techniques; facial feature localization in unconstrained videos. I improve the frame-by-frame localization results, by estimating the actual head-movement from a sequence of noisy head-pose estimates, and then using this information for detecting and fixing the localization failures.Electrical and Computer Engineerin

Teorijsko proučavanje Jahn-Teller-ovog efekta i njegovog uticaja na osobine hemijskih sistema

Quantum mechanical description of the changes in electronic structure due to distortions in molecular shape and vice versa is given in the form of the vibronic coupling theory. Probably, the most famous concept based on this theory is the Jahn−Teller (JT) effect. The JT theorem states that a molecule with a degenerate electronic state spontaneously distorts along a non-totally symmetric vibrational coordinates. This removes the degeneracy and lowers the energy. In fact, the vibronic coupling, correlation between electronic states and vibrational motion of nuclei, describes all spontaneous symmetry breaking distortions, attributed to the JT, Renner-Teller and pseudo JT effects. The consequences of the JT effect are far-reaching. JT effect affects the high magneto-resistance in manganites, superconductivity in fullerides, aromaticity, molecular stereochemistry, reactivity, magnetic properties of molecules, as well as many other properties. It should be emphasized that the JT effect has inspired very significant scientific discoveries, e.g. the concept of high-temperature superconductivity. The significance of the JT effect is increasingly recognized, hence, quantifying the distortion and getting insight into the mechanism lies at the heart of modern investigations. In this thesis, the JT effect and its consequences on the structure and properties of organic and inorganic molecules, aromaticity, excitonic coupling and excitation energy transfer are studied.The JT effect was analyzed and the JT parameters were determined for the JT active cyclobutadienyl radical cation (C4H4+•), cyclopentadienyl radical (C5H5•), benzene cation (C6H6+), benzene anion (C6H6-), tropyl radical (C7H7•), anions and cations of corannulene and coronene (C20H10- , C20H10+, C24H12- and C24H12+), small metal and metalloid clusters (Na3, Ag3, As4− , Sb4−), hexaflurocuprat(II) ion ([CuF6]4-), manganese chelate complex ([Mn(acac)3]) and the organometallic compound cobaltocene (CoCp2) by the means of...Kvantno-mehanički opis promena u elektronskoj strukturi kao posledica distorzije molekula i obrnuto dat je u formi teorije vibronske sprege. Jahn-Teller-ov (JT) efekat je verovatno najpoznatiji koncept zasnovan na ovoj teoriji. Po JT teoremi svi nelinearni molekuli sa degenerisanim elektronskim stanjem spontano se distorguju duž vibracija koje nisu totalno simetrične, pri čemu dolazi do uklanjanja degeneracije uz sniženje energije.U stvari, vibronska sprega, korelacija između elektronskih stanja i pomeraja jezgara, opisuje sve spontane distorzije molekulskih sistema i odnosi se na JT efekat, Renner-Teller-ov efekat i pseudo JT efekat. Posledice JT efekta su dalekosežne. JT efekat utiče na visoku magneto-otpornost manganita, superprovodljivost fulerida, aromatičnost, stereohemiju molekula, reaktivnost, magnetne osobine molekula, kao i mnoge druge osobine. Treba istaći da je JT efekat inspirisao veoma značajna naučna otkrića, na primer, koncept visoko-temperaturne superprovodljivosti. Značaj JT efekta se sve više prepoznaje, stoga, kvantifikovanje distorzije i dobijanje uvida u njen mehanizam se nalazi u centru modernih istraživanja. U okviru ove teze, analiziran je JT efekat i njegove posledice na strukturu i osobine organskih i neorganskih molekula, aromatičnost, ekscitonsku spregu i prenos energije ekscitacije.JT efekat je analiziran i izračunati su JT parametri za ciklobutadienil radikal katjon (C4H4+•), ciklopentadienil radikal (C5H5•), benzen katjon (C6H6+), benzen anjon (C6H6-), tropil radikal (C7H7•), anjone i katjone koranulena i koronena (C20H10- , C20H10+, C24H12- and C24H12+), male metalne i metaloidne klastere (Na3, Ag3, As4− , Sb4−), heksafluorokuprat(II) jon ([CuF6]4-), helatni kompleks mangana ([Mn(acac)3]) i organometalno jedinjenje kobaltocen (CoCp2) primenom multideterminantne teorije funkcionala gustine (eng. Density Functional Theory, DFT). Validacija multideterminantnog DFT metoda je izvedena uz sistematsko ispitivanje uticaja različitih..