Statistical Mechanics of Learning: A Variational Approach for Real Data

Using a variational technique, we generalize the statistical physics approach of learning from random examples to make it applicable to real data. We demonstrate the validity and relevance of our method by computing approximate estimators for generalization errors that are based on training data alone.Comment: 4 pages, 2 figure

Fast stable direct fitting and smoothness selection for Generalized Additive Models

Existing computationally efficient methods for penalized likelihood GAM fitting employ iterative smoothness selection on working linear models (or working mixed models). Such schemes fail to converge for a non-negligible proportion of models, with failure being particularly frequent in the presence of concurvity. If smoothness selection is performed by optimizing `whole model' criteria these problems disappear, but until now attempts to do this have employed finite difference based optimization schemes which are computationally inefficient, and can suffer from false convergence. This paper develops the first computationally efficient method for direct GAM smoothness selection. It is highly stable, but by careful structuring achieves a computational efficiency that leads, in simulations, to lower mean computation times than the schemes based on working-model smoothness selection. The method also offers a reliable way of fitting generalized additive mixed models

Study of fluid displacement in 3D porous media with an improved multi-component pseudopotential lattice Boltzmann method

We generalize to three dimensions (3D) a recently developed improved multi-component pseudopotential lattice Boltzmann method and analyze its applicability to simulate flows through realistic porous media. The model is validated and characterized via benchmarks, and we investigate its performance by simulating the displacement of immiscible fluids in 3D geometries. Two samples are considered, namely, a pack of spheres obtained numerically, and a Bentheimer sandstone rock sample obtained experimentally. We show that, with this model it is possible to simulate realistic viscosity ratios, to tune surface tension independently and, most importantly, to preserve the volume of trapped fluid. We also evaluate the computational performance of the model on the Graphical Processing Unit (GPU) and mention the implemented optimizations to increase the computational speed and reduce the memory requirements.Comment: arXiv admin note: text overlap with arXiv:2111.0866

A Bayesian Approach to Inverse Quantum Statistics

A nonparametric Bayesian approach is developed to determine quantum potentials from empirical data for quantum systems at finite temperature. The approach combines the likelihood model of quantum mechanics with a priori information over potentials implemented in form of stochastic processes. Its specific advantages are the possibilities to deal with heterogeneous data and to express a priori information explicitly, i.e., directly in terms of the potential of interest. A numerical solution in maximum a posteriori approximation was feasible for one--dimensional problems. Using correct a priori information turned out to be essential.Comment: 4 pages, 6 figures, revte

Reproducing Kernels of Generalized Sobolev Spaces via a Green Function Approach with Distributional Operators

In this paper we introduce a generalized Sobolev space by defining a semi-inner product formulated in terms of a vector distributional operator $\mathbf{P}$ consisting of finitely or countably many distributional operators $P_n$, which are defined on the dual space of the Schwartz space. The types of operators we consider include not only differential operators, but also more general distributional operators such as pseudo-differential operators. We deduce that a certain appropriate full-space Green function $G$ with respect to $L:=\mathbf{P}^{\ast T}\mathbf{P}$ now becomes a conditionally positive definite function. In order to support this claim we ensure that the distributional adjoint operator $\mathbf{P}^{\ast}$ of $\mathbf{P}$ is well-defined in the distributional sense. Under sufficient conditions, the native space (reproducing-kernel Hilbert space) associated with the Green function $G$ can be isometrically embedded into or even be isometrically equivalent to a generalized Sobolev space. As an application, we take linear combinations of translates of the Green function with possibly added polynomial terms and construct a multivariate minimum-norm interpolant $s_{f,X}$ to data values sampled from an unknown generalized Sobolev function $f$ at data sites located in some set $X \subset \mathbb{R}^d$. We provide several examples, such as Mat\'ern kernels or Gaussian kernels, that illustrate how many reproducing-kernel Hilbert spaces of well-known reproducing kernels are isometrically equivalent to a generalized Sobolev space. These examples further illustrate how we can rescale the Sobolev spaces by the vector distributional operator $\mathbf{P}$. Introducing the notion of scale as part of the definition of a generalized Sobolev space may help us to choose the "best" kernel function for kernel-based approximation methods.Comment: Update version of the publish at Num. Math. closed to Qi Ye's Ph.D. thesis (\url{http://mypages.iit.edu/~qye3/PhdThesis-2012-AMS-QiYe-IIT.pdf}

Detecting Generalized Synchronization Between Chaotic Signals: A Kernel-based Approach

A unified framework for analyzing generalized synchronization in coupled chaotic systems from data is proposed. The key of the proposed approach is the use of the kernel methods recently developed in the field of machine learning. Several successful applications are presented, which show the capability of the kernel-based approach for detecting generalized synchronization. It is also shown that the dynamical change of the coupling coefficient between two chaotic systems can be captured by the proposed approach.Comment: 20 pages, 15 figures. massively revised as a full paper; issues on the choice of parameters by cross validation, tests by surrogated data, etc. are added as well as additional examples and figure

Estimation of Regularization Parameters in Multiple-Image Deblurring

We consider the estimation of the regularization parameter for the simultaneous deblurring of multiple noisy images via Tikhonov regularization. We approach the problem in three ways. We first reduce the problem to a single-image deblurring for which the regularization parameter can be estimated through a classic generalized cross-validation (GCV) method. A modification of this function is used for correcting the undersmoothing typical of the original technique. With a second method, we minimize an average least-squares fit to the images and define a new GCV function. In the last approach, we use the classical $GCV$ on a single higher-dimensional image obtained by concatanating all the images into a single vector. With a reliable estimator of the regularization parameter, one can fully exploit the excellent computational characteristics typical of direct deblurring methods, which, especially for large images, makes them competitive with the more flexible but much slower iterative algorithms. The performance of the techniques is analyzed through numerical experiments. We find that under the independent homoscedastic and Gaussian assumptions made on the noise, the three approaches provide almost identical results with the first single image providing the practical advantage that no new software is required and the same image can be used with other deblurring algorithms.Comment: To appear in Astronomy & Astrophysic

New Approaches To Photometric Redshift Prediction Via Gaussian Process Regression In The Sloan Digital Sky Survey

Expanding upon the work of Way and Srivastava 2006 we demonstrate how the use of training sets of comparable size continue to make Gaussian process regression (GPR) a competitive approach to that of neural networks and other least-squares fitting methods. This is possible via new large size matrix inversion techniques developed for Gaussian processes (GPs) that do not require that the kernel matrix be sparse. This development, combined with a neural-network kernel function appears to give superior results for this problem. Our best fit results for the Sloan Digital Sky Survey (SDSS) Main Galaxy Sample using u,g,r,i,z filters gives an rms error of 0.0201 while our results for the same filters in the luminous red galaxy sample yield 0.0220. We also demonstrate that there appears to be a minimum number of training-set galaxies needed to obtain the optimal fit when using our GPR rank-reduction methods. We find that morphological information included with many photometric surveys appears, for the most part, to make the photometric redshift evaluation slightly worse rather than better. This would indicate that most morphological information simply adds noise from the GP point of view in the data used herein. In addition, we show that cross-match catalog results involving combinations of the Two Micron All Sky Survey, SDSS, and Galaxy Evolution Explorer have to be evaluated in the context of the resulting cross-match magnitude and redshift distribution. Otherwise one may be misled into overly optimistic conclusions.Comment: 32 pages, ApJ in Press, 2 new figures, 1 new table of comparison methods, updated discussion, references and typos to reflect version in Pres

Splines and Wavelets on Geophysically Relevant Manifolds

Analysis on the unit sphere $\mathbb{S}^{2}$ found many applications in seismology, weather prediction, astrophysics, signal analysis, crystallography, computer vision, computerized tomography, neuroscience, and statistics. In the last two decades, the importance of these and other applications triggered the development of various tools such as splines and wavelet bases suitable for the unit spheres $\mathbb{S}^{2}$, $\>\>\mathbb{S}^{3}$ and the rotation group $SO(3)$. Present paper is a summary of some of results of the author and his collaborators on generalized (average) variational splines and localized frames (wavelets) on compact Riemannian manifolds. The results are illustrated by applications to Radon-type transforms on $\mathbb{S}^{d}$ and $SO(3)$.Comment: The final publication is available at http://www.springerlink.co