Calibrated Multivariate Regression with Application to Neural Semantic Basis Discovery

We propose a calibrated multivariate regression method named CMR for fitting high dimensional multivariate regression models. Compared with existing methods, CMR calibrates regularization for each regression task with respect to its noise level so that it simultaneously attains improved finite-sample performance and tuning insensitiveness. Theoretically, we provide sufficient conditions under which CMR achieves the optimal rate of convergence in parameter estimation. Computationally, we propose an efficient smoothed proximal gradient algorithm with a worst-case numerical rate of convergence \cO(1/\epsilon), where $\epsilon$ is a pre-specified accuracy of the objective function value. We conduct thorough numerical simulations to illustrate that CMR consistently outperforms other high dimensional multivariate regression methods. We also apply CMR to solve a brain activity prediction problem and find that it is as competitive as a handcrafted model created by human experts. The R package \texttt{camel} implementing the proposed method is available on the Comprehensive R Archive Network \url{http://cran.r-project.org/web/packages/camel/}.Comment: Journal of Machine Learning Research, 201

Dimensionality Reduction for Stationary Time Series via Stochastic Nonconvex Optimization

Stochastic optimization naturally arises in machine learning. Efficient algorithms with provable guarantees, however, are still largely missing, when the objective function is nonconvex and the data points are dependent. This paper studies this fundamental challenge through a streaming PCA problem for stationary time series data. Specifically, our goal is to estimate the principle component of time series data with respect to the covariance matrix of the stationary distribution. Computationally, we propose a variant of Oja's algorithm combined with downsampling to control the bias of the stochastic gradient caused by the data dependency. Theoretically, we quantify the uncertainty of our proposed stochastic algorithm based on diffusion approximations. This allows us to prove the asymptotic rate of convergence and further implies near optimal asymptotic sample complexity. Numerical experiments are provided to support our analysis

On Prediction Properties of Kriging: Uniform Error Bounds and Robustness

Kriging based on Gaussian random fields is widely used in reconstructing unknown functions. The kriging method has pointwise predictive distributions which are computationally simple. However, in many applications one would like to predict for a range of untried points simultaneously. In this work we obtain some error bounds for the (simple) kriging predictor under the uniform metric. It works for a scattered set of input points in an arbitrary dimension, and also covers the case where the covariance function of the Gaussian process is misspecified. These results lead to a better understanding of the rate of convergence of kriging under the Gaussian or the Mat\'ern correlation functions, the relationship between space-filling designs and kriging models, and the robustness of the Mat\'ern correlation functions

The Significance of Hosea 13:11: A Study of the Monarchy in Ancient Israel

Problem In Hosea 13:11, God said that I gave you a king in My anger and took him away in My wrath. Why did God say that? What is the meaning of this expression? It is obvious that God expressed a negative view in this verse. Regarding the reason, why God held such an attitude, there are four common understandings: (1) because of the sins of the Israelites, (2) because the Israelites did not trust God in demanding a king for themselves, (3) because God wanted to give the Israelites a lesson in their disobedience, (4) because God denied the monarchy as a political system. The fourth point of view is more logical for several reasons. It is reasonable to believe that the negative view of God was not toward a certain king, as an individual, but the monarchy as a political system. Thus, the monarchy concept is essential for exploring the significance of Hosea 13:11 and a detailed study needs to be conducted. Method The study was based on the hermeneutic analysis of Hosea 13:11 and related verses, and by examining the monarchy in the context of history, politics, and theology. Results The results of this study showed that the defects and the effects of the monarchy, as a political system, were the very reasons why God expressed a negative view in Hosea 13:11. Conclusion Hosea 13:11 reflects the denial of the monarchy as a political system because God\u27s plan could not be fulfilled under the monarchy. Superficially, the monarchy was a betrayal of God and the foundations of Israel as a nation were shaken by the various system defects of the monarchy. The root cause was that according to the clues in Genesis 18:19, the goals that Israel as a nation should have practiced in God’s plan, could not be fulfilled under the monarchy. Thus, due to the monarchy, Israel as a nation failed in God\u27s plan and God\u27s negative view of the monarchy in Hosea 13:11 was reasonable

NESTT: A Nonconvex Primal-Dual Splitting Method for Distributed and Stochastic Optimization

We study a stochastic and distributed algorithm for nonconvex problems whose objective consists of a sum of $N$ nonconvex $L_i/N$-smooth functions, plus a nonsmooth regularizer. The proposed NonconvEx primal-dual SpliTTing (NESTT) algorithm splits the problem into $N$ subproblems, and utilizes an augmented Lagrangian based primal-dual scheme to solve it in a distributed and stochastic manner. With a special non-uniform sampling, a version of NESTT achieves $\epsilon$-stationary solution using $\mathcal{O}((\sum_{i=1}^N\sqrt{L_i/N})^2/\epsilon)$ gradient evaluations, which can be up to $\mathcal{O}(N)$ times better than the (proximal) gradient descent methods. It also achieves Q-linear convergence rate for nonconvex $\ell_1$ penalized quadratic problems with polyhedral constraints. Further, we reveal a fundamental connection between primal-dual based methods and a few primal only methods such as IAG/SAG/SAGA.Comment: 35 pages, 2 figure

Solid-state NMR investigations of cellulose structure and interactions with matrix polysaccharides in plant primary cell walls

Until recently, the 3D architecture of plant cell walls was poorly understood due to the lack of high-resolution techniques for characterizing the molecular structure, dynamics, and intermolecular interactions of the wall polysaccharides in these insoluble biomolecular mixtures. We introduced multidimensional solid-state NMR (SSNMR) spectroscopy, coupled with [superscript 13]C labelling of whole plants, to determine the spatial arrangements of macromolecules in near-native plant cell walls. Here we review key evidence from 2D and 3D correlation NMR spectra that show relatively few cellulose–hemicellulose cross peaks but many cellulose–pectin cross peaks, indicating that cellulose microfibrils are not extensively coated by hemicellulose and all three major polysaccharides exist in a single network rather than two separate networks as previously proposed. The number of glucan chains in the primary-wall cellulose microfibrils has been under active debate recently. We show detailed analysis of quantitative [superscript 13]C SSNMR spectra of cellulose in various wild-type (WT) and mutant Arabidopsis and Brachypodium primary cell walls, which consistently indicate that primary-wall cellulose microfibrils contain at least 24 glucan chains.United States. Dept. of Energy. Office of Basic Energy Sciences (Center for Lignocellulose Structure and Formation Award # DE-SC0001090