Data complexity in machine learning

We investigate the role of data complexity in the context of binary classification problems. The universal data complexity is defined for a data set as the Kolmogorov complexity of the mapping enforced by the data set. It is closely related to several existing principles used in machine learning such as Occam's razor, the minimum description length, and the Bayesian approach. The data complexity can also be defined based on a learning model, which is more realistic for applications. We demonstrate the application of the data complexity in two learning problems, data decomposition and data pruning. In data decomposition, we illustrate that a data set is best approximated by its principal subsets which are Pareto optimal with respect to the complexity and the set size. In data pruning, we show that outliers usually have high complexity contributions, and propose methods for estimating the complexity contribution. Since in practice we have to approximate the ideal data complexity measures, we also discuss the impact of such approximations

A frequency-domain approach to the analysis of stability and bifurcations in nonlinear systems described by differential-algebraic equations

A general numerical technique is proposed for the assessment of the stability of periodic solutions and the determination of bifurcations for limit cycles in autonomous nonlinear systems represented by ordinary differential equations in the differential-algebraic form. The method is based on the harmonic balance technique, and exploits the same Jacobian matrix of the nonlinear system used in the Newton iterative numerical solution of the harmonic balance equations for the determination of the periodic steady-state. To demonstrate the approach, it is applied to the determination of the bifurcation curves in the parameters' space of Chua's circuit with cubic nonlinearity, and to study the dynamics of the limit cycle of a Colpitts oscillato

Numerical modeling of black holes as sources of gravitational waves in a nutshell

These notes summarize basic concepts underlying numerical relativity and in particular the numerical modeling of black hole dynamics as a source of gravitational waves. Main topics are the 3+1 decomposition of general relativity, the concept of a well-posed initial value problem, the construction of initial data for general relativity, trapped surfaces and gravitational waves. Also, a brief summary is given of recent progress regarding the numerical evolution of black hole binary systems.Comment: 28 pages, lectures given at winter school 'Conceptual and Numerical Challenges in Femto- and Peta-Scale Physics' in Schladming, Austria, 200

A Distributed Newton Method for Network Utility Maximization

Most existing work uses dual decomposition and subgradient methods to solve Network Utility Maximization (NUM) problems in a distributed manner, which suffer from slow rate of convergence properties. This work develops an alternative distributed Newton-type fast converging algorithm for solving network utility maximization problems with self-concordant utility functions. By using novel matrix splitting techniques, both primal and dual updates for the Newton step can be computed using iterative schemes in a decentralized manner with limited information exchange. Similarly, the stepsize can be obtained via an iterative consensus-based averaging scheme. We show that even when the Newton direction and the stepsize in our method are computed within some error (due to finite truncation of the iterative schemes), the resulting objective function value still converges superlinearly to an explicitly characterized error neighborhood. Simulation results demonstrate significant convergence rate improvement of our algorithm relative to the existing subgradient methods based on dual decomposition.Comment: 27 pages, 4 figures, LIDS report, submitted to CDC 201