Learning Kernel-Based Halfspaces with the Zero-One Loss

We describe and analyze a new algorithm for agnostically learning kernel-based halfspaces with respect to the \emph{zero-one} loss function. Unlike most previous formulations which rely on surrogate convex loss functions (e.g. hinge-loss in SVM and log-loss in logistic regression), we provide finite time/sample guarantees with respect to the more natural zero-one loss function. The proposed algorithm can learn kernel-based halfspaces in worst-case time \poly(\exp(L\log(L/\epsilon))), for \emph{any} distribution, where $L$ is a Lipschitz constant (which can be thought of as the reciprocal of the margin), and the learned classifier is worse than the optimal halfspace by at most $\epsilon$. We also prove a hardness result, showing that under a certain cryptographic assumption, no algorithm can learn kernel-based halfspaces in time polynomial in $L$.Comment: This is a full version of the paper appearing in the 23rd International Conference on Learning Theory (COLT 2010). Compared to the previous arXiv version, this version contains some small corrections in the proof of Lemma 3 and in appendix

On the constrained mock-Chebyshev least-squares

The algebraic polynomial interpolation on uniformly distributed nodes is affected by the Runge phenomenon, also when the function to be interpolated is analytic. Among all techniques that have been proposed to defeat this phenomenon, there is the mock-Chebyshev interpolation which is an interpolation made on a subset of the given nodes whose elements mimic as well as possible the Chebyshev-Lobatto points. In this work we use the simultaneous approximation theory to combine the previous technique with a polynomial regression in order to increase the accuracy of the approximation of a given analytic function. We give indications on how to select the degree of the simultaneous regression in order to obtain polynomial approximant good in the uniform norm and provide a sufficient condition to improve, in that norm, the accuracy of the mock-Chebyshev interpolation with a simultaneous regression. Numerical results are provided.Comment: 17 pages, 9 figure

A fast and well-conditioned spectral method for singular integral equations

We develop a spectral method for solving univariate singular integral equations over unions of intervals by utilizing Chebyshev and ultraspherical polynomials to reformulate the equations as almost-banded infinite-dimensional systems. This is accomplished by utilizing low rank approximations for sparse representations of the bivariate kernels. The resulting system can be solved in ${\cal O}(m^2n)$ operations using an adaptive QR factorization, where $m$ is the bandwidth and $n$ is the optimal number of unknowns needed to resolve the true solution. The complexity is reduced to ${\cal O}(m n)$ operations by pre-caching the QR factorization when the same operator is used for multiple right-hand sides. Stability is proved by showing that the resulting linear operator can be diagonally preconditioned to be a compact perturbation of the identity. Applications considered include the Faraday cage, and acoustic scattering for the Helmholtz and gravity Helmholtz equations, including spectrally accurate numerical evaluation of the far- and near-field solution. The Julia software package SingularIntegralEquations.jl implements our method with a convenient, user-friendly interface

Numerical approximations for population growth model by Rational Chebyshev and Hermite Functions collocation approach: A comparison

This paper aims to compare rational Chebyshev (RC) and Hermite functions (HF) collocation approach to solve the Volterra's model for population growth of a species within a closed system. This model is a nonlinear integro-differential equation where the integral term represents the effect of toxin. This approach is based on orthogonal functions which will be defined. The collocation method reduces the solution of this problem to the solution of a system of algebraic equations. We also compare these methods with some other numerical results and show that the present approach is applicable for solving nonlinear integro-differential equations.Comment: 18 pages, 5 figures; Published online in the journal of "Mathematical Methods in the Applied Sciences

An algorithm for the rapid numerical evaluation of Bessel functions of real orders and arguments

We describe a method for the rapid numerical evaluation of the Bessel functions of the first and second kinds of nonnegative real orders and positive arguments. Our algorithm makes use of the well-known observation that although the Bessel functions themselves are expensive to represent via piecewise polynomial expansions, the logarithms of certain solutions of Bessel's equation are not. We exploit this observation by numerically precomputing the logarithms of carefully chosen Bessel functions and representing them with piecewise bivariate Chebyshev expansions. Our scheme is able to evaluate Bessel functions of orders between $0$ and 1\sep,000\sep,000\sep,000 at essentially any positive real argument. In that regime, it is competitive with existing methods for the rapid evaluation of Bessel functions and has several advantages over them. First, our approach is quite general and can be readily applied to many other special functions which satisfy second order ordinary differential equations. Second, by calculating the logarithms of the Bessel functions rather than the Bessel functions themselves, we avoid many issues which arise from numerical overflow and underflow. Third, in the oscillatory regime, our algorithm calculates the values of a nonoscillatory phase function for Bessel's differential equation and its derivative. These quantities are useful for computing the zeros of Bessel functions, as well as for rapidly applying the Fourier-Bessel transform. The results of extensive numerical experiments demonstrating the efficacy of our algorithm are presented. A Fortran package which includes our code for evaluating the Bessel functions as well as our code for all of the numerical experiments described here is publically available

Fast, numerically stable computation of oscillatory integrals with stationary points

We present a numerically stable way to compute oscillatory integrals of the form $\int{-1}^{1} f(x)e^{i\omega g(x)}dx$. For each additional frequency, only a small, well-conditioned linear system with a Hessenberg matrix must be solved, and the amount of work needed decreases as the frequency increases. Moreover, we can modify the method for computing oscillatory integrals with stationary points. This is the first stable algorithm for oscillatory integrals with stationary points which does not lose accuracy as the frequency increases and does not require deformation into the complex plane