Training Support Vector Machines Using Frank-Wolfe Optimization Methods

Training a Support Vector Machine (SVM) requires the solution of a quadratic programming problem (QP) whose computational complexity becomes prohibitively expensive for large scale datasets. Traditional optimization methods cannot be directly applied in these cases, mainly due to memory restrictions. By adopting a slightly different objective function and under mild conditions on the kernel used within the model, efficient algorithms to train SVMs have been devised under the name of Core Vector Machines (CVMs). This framework exploits the equivalence of the resulting learning problem with the task of building a Minimal Enclosing Ball (MEB) problem in a feature space, where data is implicitly embedded by a kernel function. In this paper, we improve on the CVM approach by proposing two novel methods to build SVMs based on the Frank-Wolfe algorithm, recently revisited as a fast method to approximate the solution of a MEB problem. In contrast to CVMs, our algorithms do not require to compute the solutions of a sequence of increasingly complex QPs and are defined by using only analytic optimization steps. Experiments on a large collection of datasets show that our methods scale better than CVMs in most cases, sometimes at the price of a slightly lower accuracy. As CVMs, the proposed methods can be easily extended to machine learning problems other than binary classification. However, effective classifiers are also obtained using kernels which do not satisfy the condition required by CVMs and can thus be used for a wider set of problems

Reduced basis method for source mask optimization

Image modeling and simulation are critical to extending the limits of leading edge lithography technologies used for IC making. Simultaneous source mask optimization (SMO) has become an important objective in the field of computational lithography. SMO is considered essential to extending immersion lithography beyond the 45nm node. However, SMO is computationally extremely challenging and time-consuming. The key challenges are due to run time vs. accuracy tradeoffs of the imaging models used for the computational lithography. We present a new technique to be incorporated in the SMO flow. This new approach is based on the reduced basis method (RBM) applied to the simulation of light transmission through the lithography masks. It provides a rigorous approximation to the exact lithographical problem, based on fully vectorial Maxwell's equations. Using the reduced basis method, the optimization process is divided into an offline and an online steps. In the offline step, a RBM model with variable geometrical parameters is built self-adaptively and using a Finite Element (FEM) based solver. In the online step, the RBM model can be solved very fast for arbitrary illumination and geometrical parameters, such as dimensions of OPC features, line widths, etc. This approach dramatically reduces computational costs of the optimization procedure while providing accuracy superior to the approaches involving simplified mask models. RBM furthermore provides rigorous error estimators, which assure the quality and reliability of the reduced basis solutions. We apply the reduced basis method to a 3D SMO example. We quantify performance, computational costs and accuracy of our method.Comment: BACUS Photomask Technology 201

Parallel decomposition methods for linearly constrained problems subject to simple bound with application to the SVMs training

We consider the convex quadratic linearly constrained problem with bounded variables and with huge and dense Hessian matrix that arises in many applications such as the training problem of bias support vector machines. We propose a decomposition algorithmic scheme suitable to parallel implementations and we prove global convergence under suitable conditions. Focusing on support vector machines training, we outline how these assumptions can be satisfied in practice and we suggest various specific implementations. Extensions of the theoretical results to general linearly constrained problem are provided. We included numerical results on support vector machines with the aim of showing the viability and the effectiveness of the proposed scheme

Thermodynamic analysis of the Quantum Critical behavior of Ce-lattice compounds

A systematic analysis of low temperature magnetic phase diagrams of Ce compounds is performed in order to recognize the thermodynamic conditions to be fulfilled by those systems to reach a quantum critical regime and, alternatively, to identify other kinds of low temperature behaviors. Based on specific heat ($C_m$) and entropy ($S_m$) results, three different types of phase diagrams are recognized: i) with the entropy involved into the ordered phase ($S_{MO}$) decreasing proportionally to the ordering temperature ($T_{MO}$), ii) those showing a transference of degrees of freedom from the ordered phase to a non-magnetic component, with their $C_m(T_{MO})$ jump ($\Delta C_m$) vanishing at finite temperature, and iii) those ending in a critical point at finite temperature because their $\Delta C_m$ do not decrease with $T_{MO}$ producing an entropy accumulation at low temperature. Only those systems belonging to the first case, i.e. with $S_{MO}\to 0$ as $T_{MO}\to 0$, can be regarded as candidates for quantum critical behavior. Their magnetic phase boundaries deviate from the classical negative curvature below $T\approx 2.5$\,K, denouncing frequent misleading extrapolations down to T=0. Different characteristic concentrations are recognized and analyzed for Ce-ligand alloyed systems. Particularly, a pre-critical region is identified, where the nature of the magnetic transition undergoes significant modifications, with its $\partial C_m/\partial T$ discontinuity strongly affected by magnetic field and showing an increasing remnant entropy at $T\to 0$. Physical constraints arising from the third law at $T\to 0$ are discussed and recognized from experimental results