Karhunen-Loeve expansions and their applications.

The Karhunen-Loeve Expansion (K-L expansion) is a bi-orthogonal stochastic process expansion. In the field of stochastic process, the Karhunen-Loeve expansion decomposes the process into a series of orthogonal functions with the random coefficients. The essential idea of the expansion is to solve the Fredholm integral equation, associated with the covariance kernel of the process, which defines a Reproducing Kernel Hilbert Space (RKHS). This either has an analytical solution or special numerical methods are needed. This thesis applies the Karhunen-Loeve expansion to some fields of statistics. The first two chapters review the theoretical background of the Karhunen-Loeve expansion and introduce the numerical methods, including the integral method and the expansion method, when the analytical solution to the expansion is unavailable. Chapter 3 applies the theory of the Karhunen-Loeve expansion to the field of the design experiment using a criteria called "maximum entropy sampling". Under such setting, a type of duality is set up between maximum entropy sampling and the D- optimal design of the classical optimal design. Chapter 4 uses the Karhunen-Loeve expansion to calculate the conditional mean and variance for a given set of observations, with application to prediction. Chapter 5 extends the theory of the Karhunen- Loeve expansion from the univariate setting to the multivariate setting: multivariate space, univariate time. Adaptations of numerical methods of Chapter 2 are also provided for the multivariate setting, with a full matrix development. Chapter 6 applies the numerical method developed in Chapter 5 to the emerging area of multivariate functional data analysis with a detailed example on a trivariate autoregressive process

The Large Deviation Principle and Steady-state Fluctuation Theorem for the Entropy Production Rate of a Stochastic Process in Magnetic Fields

Fluctuation theorem is one of the major achievements in the field of nonequilibrium statistical mechanics during the past two decades. Steady-state fluctuation theorem of sample entropy production rate in terms of large deviation principle for diffusion processes have not been rigorously proved yet due to technical difficulties. Here we give a proof for the steady-state fluctuation theorem of a diffusion process in magnetic fields, with explicit expressions of the free energy function and rate function. The proof is based on the Karhunen-Lo\'{e}ve expansion of complex-valued Ornstein-Uhlenbeck process

Expansions of Iterated Stratonovich Stochastic Integrals of Multiplicities 1 to 4 Based on Generalized Multiple Fourier Series

The article is devoted to the expansions of iterated Stratonovich stochastic integrals of multiplicities 1 to 4 on the basis of the method of generalized multiple Fourier series that are converge in the sense of norm in Hilbert space $L_2([t, T]^k),$ $k=1,2,3,4.$ Mean-square convergence of the expansions for the case of multiple Fourier-Legendre series and for the case of multiple trigonometric Fourier series is proved. The considered expansions contain only one operation of the limit transition in contrast to its existing analogues. This property is very important for the mean-square approximation of iterated stochastic integrals. The results of the article can be applied to numerical integration of Ito stochastic differential equations with multidimensional non-commutative noises.Comment: 50 pages. Sect. 6 is added. Minor changes along the article in the whole. arXiv admin note: text overlap with arXiv:1712.0899

Response to Comments on PCA Based Hurst Exponent Estimator for fBm Signals Under Disturbances

In this response, we try to give a repair to our previous proof for PCA Based Hurst Exponent Estimator for fBm Signals by using orthogonal projection. Moreover, we answer the question raised recently: If a centered Gaussian process $G_t$ admits two series expansions on different Riesz bases, we may possibly study the asymptotic behavior of one eigenvalue sequence from the knowledge on the asymptotic behaviors of another.Comment: This is a response for a mistake in Li Li, Jianming Hu, Yudong Chen, Yi Zhang, PCA based Hurst exponent estimator for fBm signals under disturbances, IEEE Transactions on Signal Processing, vol. 57, no. 7, pp. 2840-2846, 200

An efficient polynomial chaos-based proxy model for history matching and uncertainty quantification of complex geological structures

A novel polynomial chaos proxy-based history matching and uncertainty quantification method is presented that can be employed for complex geological structures in inverse problems. For complex geological structures, when there are many unknown geological parameters with highly nonlinear correlations, typically more than 106 full reservoir simulation runs might be required to accurately probe the posterior probability space given the production history of reservoir. This is not practical for high-resolution geological models. One solution is to use a "proxy model" that replicates the simulation model for selected input parameters. The main advantage of the polynomial chaos proxy compared to other proxy models and response surfaces is that it is generally applicable and converges systematically as the order of the expansion increases. The Cameron and Martin theorem 2.24 states that the convergence rate of the standard polynomial chaos expansions is exponential for Gaussian random variables. To improve the convergence rate for non-Gaussian random variables, the generalized polynomial chaos is implemented that uses an Askey-scheme to choose the optimal basis for polynomial chaos expansions [199]. Additionally, for the non-Gaussian distributions that can be effectively approximated by a mixture of Gaussian distributions, we use the mixture-modeling based clustering approach where under each cluster the polynomial chaos proxy converges exponentially fast and the overall posterior distribution can be estimated more efficiently using different polynomial chaos proxies. The main disadvantage of the polynomial chaos proxy is that for high-dimensional problems, the number of the polynomial chaos terms increases drastically as the order of the polynomial chaos expansions increases. Although different non-intrusive methods have been developed in the literature to address this issue, still a large number of simulation runs is required to compute high-order terms of the polynomial chaos expansions. This work resolves this issue by proposing the reduced-terms polynomial chaos expansion which preserves only the relevant terms in the polynomial chaos representation. We demonstrated that the sparsity pattern in the polynomial chaos expansion, when used with the Karhunen-Loéve decomposition method or kernel PCA, can be systematically captured. A probabilistic framework based on the polynomial chaos proxy is also suggested in the context of the Bayesian model selection to study the plausibility of different geological interpretations of the sedimentary environments. The proposed surrogate-accelerated Bayesian inverse analysis can be coherently used in practical reservoir optimization workflows and uncertainty assessments