210 research outputs found

    Difference of Convex Functions Programming Applied to Control with Expert Data

    Get PDF
    This paper reports applications of Difference of Convex functions (DC) programming to Learning from Demonstrations (LfD) and Reinforcement Learning (RL) with expert data. This is made possible because the norm of the Optimal Bellman Residual (OBR), which is at the heart of many RL and LfD algorithms, is DC. Improvement in performance is demonstrated on two specific algorithms, namely Reward-regularized Classification for Apprenticeship Learning (RCAL) and Reinforcement Learning with Expert Demonstrations (RLED), through experiments on generic Markov Decision Processes (MDP), called Garnets

    DC Proximal Newton for Non-Convex Optimization Problems

    Get PDF
    We introduce a novel algorithm for solving learning problems where both the loss function and the regularizer are non-convex but belong to the class of difference of convex (DC) functions. Our contribution is a new general purpose proximal Newton algorithm that is able to deal with such a situation. The algorithm consists in obtaining a descent direction from an approximation of the loss function and then in performing a line search to ensure sufficient descent. A theoretical analysis is provided showing that the iterates of the proposed algorithm {admit} as limit points stationary points of the DC objective function. Numerical experiments show that our approach is more efficient than current state of the art for a problem with a convex loss functions and non-convex regularizer. We have also illustrated the benefit of our algorithm in high-dimensional transductive learning problem where both loss function and regularizers are non-convex

    Enhanced protein fold recognition through a novel data integration approach

    Get PDF
    <p>Abstract</p> <p>Background</p> <p>Protein fold recognition is a key step in protein three-dimensional (3D) structure discovery. There are multiple fold discriminatory data sources which use physicochemical and structural properties as well as further data sources derived from local sequence alignments. This raises the issue of finding the most efficient method for combining these different informative data sources and exploring their relative significance for protein fold classification. Kernel methods have been extensively used for biological data analysis. They can incorporate separate fold discriminatory features into kernel matrices which encode the similarity between samples in their respective data sources.</p> <p>Results</p> <p>In this paper we consider the problem of integrating multiple data sources using a kernel-based approach. We propose a novel information-theoretic approach based on a Kullback-Leibler (KL) divergence between the output kernel matrix and the input kernel matrix so as to integrate heterogeneous data sources. One of the most appealing properties of this approach is that it can easily cope with multi-class classification and multi-task learning by an appropriate choice of the output kernel matrix. Based on the position of the output and input kernel matrices in the KL-divergence objective, there are two formulations which we respectively refer to as <it>MKLdiv-dc </it>and <it>MKLdiv-conv</it>. We propose to efficiently solve MKLdiv-dc by a difference of convex (DC) programming method and MKLdiv-conv by a projected gradient descent algorithm. The effectiveness of the proposed approaches is evaluated on a benchmark dataset for protein fold recognition and a yeast protein function prediction problem.</p> <p>Conclusion</p> <p>Our proposed methods MKLdiv-dc and MKLdiv-conv are able to achieve state-of-the-art performance on the SCOP PDB-40D benchmark dataset for protein fold prediction and provide useful insights into the relative significance of informative data sources. In particular, MKLdiv-dc further improves the fold discrimination accuracy to 75.19% which is a more than 5% improvement over competitive Bayesian probabilistic and SVM margin-based kernel learning methods. Furthermore, we report a competitive performance on the yeast protein function prediction problem.</p

    Supervised Classification and Mathematical Optimization

    Get PDF
    Data Mining techniques often ask for the resolution of optimization problems. Supervised Classification, and, in particular, Support Vector Machines, can be seen as a paradigmatic instance. In this paper, some links between Mathematical Optimization methods and Supervised Classification are emphasized. It is shown that many different areas of Mathematical Optimization play a central role in off-the-shelf Supervised Classification methods. Moreover, Mathematical Optimization turns out to be extremely useful to address important issues in Classification, such as identifying relevant variables, improving the interpretability of classifiers or dealing with vagueness/noise in the data

    Supervised classification and mathematical optimization

    Get PDF
    Data Mining techniques often ask for the resolution of optimization problems. Supervised Classification, and, in particular, Support Vector Machines, can be seen as a paradigmatic instance. In this paper, some links between Mathematical Optimization methods and Supervised Classification are emphasized. It is shown that many different areas of Mathematical Optimization play a central role in off-the-shelf Supervised Classification methods. Moreover, Mathematical Optimization turns out to be extremely useful to address important issues in Classification, such as identifying relevant variables, improving the interpretability of classifiers or dealing with vagueness/noise in the data.Ministerio de Ciencia e InnovaciĂłnJunta de AndalucĂ­

    Difference of Convex Functions Programming Applied to Control with Expert Data

    Get PDF
    This paper reports applications of Difference of Convex functions (DC) programming to Learning from Demonstrations (LfD) and Reinforcement Learning (RL) with expert data. This is made possible because the norm of the Optimal Bellman Residual (OBR), which is at the heart of many RL and LfD algorithms, is DC. Improvement in performance is demonstrated on two specific algorithms, namely Reward-regularized Classification for Apprenticeship Learning (RCAL) and Reinforcement Learning with Expert Demonstrations (RLED), through experiments on generic Markov Decision Processes (MDP), called Garnets

    Advances in knowledge discovery and data mining Part II

    Get PDF
    19th Pacific-Asia Conference, PAKDD 2015, Ho Chi Minh City, Vietnam, May 19-22, 2015, Proceedings, Part II</p

    Bundle methods in nonsmooth DC optimization

    Get PDF
    Due to the complexity of many practical applications, we encounter optimization problems with nonsmooth functions, that is, functions which are not continuously differentiable everywhere. Classical gradient-based methods are not applicable to solve such problems, since they may fail in the nonsmooth setting. Therefore, it is imperative to develop numerical methods specifically designed for nonsmooth optimization. To date, bundle methods are considered to be the most efficient and reliable general purpose solvers for this type of problems. The idea in bundle methods is to approximate the subdifferential of the objective function by a bundle of subgradients. This information is then used to build a model for the objective. However, this model is typically convex and, due to this, it may be inaccurate and unable to adequately reflect the behaviour of the objective function in the nonconvex case. These circumstances motivate to design new bundle methods based on nonconvex models of the objective function. In this dissertation, the main focus is on nonsmooth DC optimization that constitutes an important and broad subclass of nonconvex optimization problems. A DC function can be presented as a difference of two convex functions. Thus, we can obtain a model that utilizes explicitly both the convexity and concavity of the objective by approximating separately the convex and concave parts. This way we end up with a nonconvex DC model describing the problem more accurately than the convex one. Based on the new DC model we introduce three different bundle methods. Two of them are designed for unconstrained DC optimization and the third one is capable of solving also multiobjective and constrained DC problems. The finite convergence is proved for each method. The numerical results demonstrate the efficiency of the methods and show the benefits obtained from the utilization of the DC decomposition. Even though the usage of the DC decomposition can improve the performance of the bundle methods, it is not always available or possible to construct. Thus, we present another bundle method for a general objective function implicitly collecting information about the DC structure. This method is developed for large-scale nonsmooth optimization and its convergence is proved for semismooth functions. The efficiency of the method is shown with numerical results. As an application of the developed methods, we consider the clusterwise linear regression (CLR) problems. By applying the support vector machines (SVM) approach a new model for these problems is proposed. The objective in the new formulation of the CLR problem is expressed as a DC function and a method based on one of the presented bundle methods is designed to solve it. Numerical results demonstrate robustness of the new approach to outliers.Monissa kÀytÀnnön sovelluksissa tarkastelun kohteena oleva ongelma on monimutkainen ja joudutaan nÀin ollen mallintamaan epÀsileillÀ funktioilla, jotka eivÀt vÀlttÀmÀttÀ ole jatkuvasti differentioituvia kaikkialla. Klassisia gradienttiin perustuvia optimointimenetelmiÀ ei voida kÀyttÀÀ epÀsileisiin tehtÀviin, sillÀ epÀsileillÀ funktioilla ei ole olemassa klassista gradienttia kaikkialla. NÀin ollen epÀsileÀÀn optimointiin on vÀlttÀmÀtöntÀ kehittÀÀ omia numeerisia ratkaisumenetelmiÀ. NÀistÀ kimppumenetelmiÀ pidetÀÀn tÀllÀ hetkellÀ kaikista tehokkaimpina ja luotettavimpina yleismenetelminÀ kyseisten tehtÀvien ratkaisemiseksi. Ideana kimppumenetelmissÀ on approksimoida kohdefunktion alidifferentiaalia kimpulla, joka on muodostettu kerÀÀmÀllÀ kohdefunktion aligradientteja edellisiltÀ iteraatiokierroksilta. TÀtÀ tietoa hyödyntÀmÀllÀ voidaan muodostaa kohdefunktiolle malli, joka on alkuperÀistÀ tehtÀvÀÀ helpompi ratkaista. KÀytetty malli on tyypillisesti konveksi ja nÀin ollen se voi olla epÀtarkka ja kykenemÀtön esittÀmÀÀn alkuperÀisen tehtÀvÀn rakennetta epÀkonveksissa tapauksessa. TÀstÀ syystÀ vÀitöskirjassa keskitytÀÀn kehittÀmÀÀn uusia kimppumenetelmiÀ, jotka mallinnusvaiheessa muodostavat kohdefunktiolle epÀkonveksin mallin. PÀÀpaino vÀitöskirjassa on epÀsileissÀ optimointitehtÀvissÀ, joissa funktiot voidaan esittÀÀ kahden konveksin funktion erotuksena (difference of two convex functions). KyseisiÀ funktioita kutsutaan DC-funktioiksi ja ne muodostavat tÀrkeÀn ja laajan epÀkonveksien funktioiden osajoukon. TÀmÀ valinta mahdollistaa kohdefunktion konveksisuuden ja konkaavisuuden eksplisiittisen hyödyntÀmisen, sillÀ uusi malli kohdefunktiolle muodostetaan yhdistÀmÀllÀ erilliset konveksille ja konkaaville osalle rakennetut mallit. TÀllÀ tavalla pÀÀdytÀÀn epÀkonveksiin DC-malliin, joka pystyy kuvaamaan ratkaistavaa tehtÀvÀÀ tarkemmin kuin konveksi arvio. VÀitöskirjassa esitetÀÀn kolme erilaista uuden DC-mallin pohjalta kehitettyÀ kimppumenetelmÀÀ sekÀ todistetaan menetelmien konvergenssit. Kaksi nÀistÀ menetelmistÀ on suunniteltu rajoitteettomaan DC-optimointiin ja kolmannella voidaan ratkaista myös monitavoitteisia ja rajoitteellisia DC-optimointitehtÀviÀ. Numeeriset tulokset havainnollistavat menetelmien tehokkuutta sekÀ DC-hajotelman kÀytöstÀ saatuja etuja. Vaikka DC-hajotelman kÀyttö voi parantaa kimppumenetelmien suoritusta, sitÀ ei aina ole saatavilla tai mahdollista muodostaa. TÀstÀ syystÀ vÀitöskirjassa esitetÀÀn myös neljÀs kimppumenetelmÀ konvergenssitodistuksineen yleiselle kohdefunktiolle, jossa kerÀtÀÀn implisiittisesti tietoa kohdefunktion DC-rakenteesta. MenetelmÀ on kehitetty erityisesti suurille epÀsileille optimointitehtÀville ja sen tehokkuus osoitetaan numeerisella testauksella Sovelluksena vÀitöskirjassa tarkastellaan datalle klustereittain tehtÀvÀÀ lineaarista regressiota (clusterwise linear regression). Kyseiselle sovellukselle muodostetaan uusi malli hyödyntÀen koneoppimisessa kÀytettyÀ SVM-lÀhestymistapaa (support vector machines approach) ja saatu kohdefunktio esitetÀÀn DC-funktiona. NÀin ollen yhtÀ kehitetyistÀ kimppumenetelmistÀ sovelletaan tehtÀvÀn ratkaisemiseen. Numeeriset tulokset havainnollistavat uuden lÀhestymistavan robustisuutta ja tehokkuutta
    • 

    corecore