Generalization error for multi-class margin classification

In this article, we study rates of convergence of the generalization error of multi-class margin classifiers. In particular, we develop an upper bound theory quantifying the generalization error of various large margin classifiers. The theory permits a treatment of general margin losses, convex or nonconvex, in presence or absence of a dominating class. Three main results are established. First, for any fixed margin loss, there may be a trade-off between the ideal and actual generalization performances with respect to the choice of the class of candidate decision functions, which is governed by the trade-off between the approximation and estimation errors. In fact, different margin losses lead to different ideal or actual performances in specific cases. Second, we demonstrate, in a problem of linear learning, that the convergence rate can be arbitrarily fast in the sample size $n$ depending on the joint distribution of the input/output pair. This goes beyond the anticipated rate $O(n^{-1})$. Third, we establish rates of convergence of several margin classifiers in feature selection with the number of candidate variables $p$ allowed to greatly exceed the sample size $n$ but no faster than $\exp(n)$.Comment: Published at http://dx.doi.org/10.1214/07-EJS069 in the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics (http://www.imstat.org

Dynamic analysis and program formalization of regional banks activity in catastrophe theory conception

Present researches in the framework of catastrophe theory aimed to describe in formalized form the dynamics of regional banks activity in transitive economics. For this purpose the system of differential equations was algorithmically solved and the decision with approximation error in 9.4% gives reliable description of the real statistics of banks activity. Certain managing parameters, establishing the dynamics of money input components interaction and output producing in banks activity were defined and theoretical approach to functional dependencies finding of above mentioned parameters was made. The integral activity curve of the bank was build and it showed the exact time Tx of bifurcation appearance, bringing about the misbalance in banks production with consequence in banks catastrophe. It was found that since the bank integral activity curve had fallen below the zero the bank experienced the "fold" bifurcation, assuming the catastrophe( bankruptcy) and this situation couldn''''t be diagnosed in time by means of traditional methods of bank analysis. The ways of planning and crisis management in regional politics conception were offered by varying of initial banks inputs and managing parameters.

Dynamic financial processes identification using sparse regressive reservoir computers

In this document, we present key findings in structured matrix approximation theory, with applications to the regressive representation of dynamic financial processes. Initially, we explore a comprehensive approach involving generic nonlinear time delay embedding for time series data extracted from a financial or economic system under examination. Subsequently, we employ sparse least-squares and structured matrix approximation methods to discern approximate representations of the output coupling matrices. These representations play a pivotal role in establishing the regressive models corresponding to the recursive structures inherent in a given financial system. The document further introduces prototypical algorithms that leverage the aforementioned techniques. These algorithms are demonstrated through applications in approximate identification and predictive simulation of dynamic financial and economic processes, encompassing scenarios that may or may not exhibit chaotic behavior.Comment: The content of this publication represents the opinion of the researchers affiliated with the Department of Statistics and Research, but not the official opinion of the CNB

Bit Error Rates for Ultrafast APD Based Optical Receivers: Exact and Large Deviation Based Asymptotic Approaches

Exact analysis as well as asymptotic analysis, based on large-deviation theory (LDT), are developed to compute the bit-error rate (BER) for ultrafast avalanche-photodiode (APD) based optical receivers assuming on-off keying and direct detection. The effects of intersymbol interference (ISI), resulting from the APD\u27s stochastic avalanche buildup time, as well as the APD\u27s dead space are both included in the analysis. ISI becomes a limiting factor as the transmission rate approaches the detector\u27s bandwidth, in which case the bit duration becomes comparable to APD\u27s avalanche buildup time. Further, the effect of dead space becomes significant in high-speed APDs that employ thin avalanche multiplication regions. While the exact BER analysis at the generality considered here has not been reported heretofore, the asymptotic analysis is a major generalization of that developed by Letaief and Sadowsky [IEEE Trans. Inform. Theory, vol. 38, 1992], in which the LDT was used to estimate the BER assuming APDs with an instantaneous response (negligible avalanche buildup time) and no dead space. These results are compared with those obtained using the common Gaussian approximation approach showing the inadequacy of the Guassian approximation when ISI noise has strong presence

Asymptotics of Transmit Antenna Selection: Impact of Multiple Receive Antennas

Consider a fading Gaussian MIMO channel with $N_\mathrm{t}$ transmit and $N_\mathrm{r}$ receive antennas. The transmitter selects $L_\mathrm{t}$ antennas corresponding to the strongest channels. For this setup, we study the distribution of the input-output mutual information when $N_\mathrm{t}$ grows large. We show that, for any $N_\mathrm{r}$ and $L_\mathrm{t}$, the distribution of the input-output mutual information is accurately approximated by a Gaussian distribution whose mean grows large and whose variance converges to zero. Our analysis depicts that, in the large limit, the gap between the expectation of the mutual information and its corresponding upper bound, derived by applying Jensen's inequality, converges to a constant which only depends on $N_\mathrm{r}$ and $L_\mathrm{t}$. The result extends the scope of channel hardening to the general case of antenna selection with multiple receive and selected transmit antennas. Although the analyses are given for the large-system limit, our numerical investigations indicate the robustness of the approximated distribution even when the number of antennas is not large.Comment: 6 pages, 4 figures, ICC 201

Approximate Computation and Implicit Regularization for Very Large-scale Data Analysis

Database theory and database practice are typically the domain of computer scientists who adopt what may be termed an algorithmic perspective on their data. This perspective is very different than the more statistical perspective adopted by statisticians, scientific computers, machine learners, and other who work on what may be broadly termed statistical data analysis. In this article, I will address fundamental aspects of this algorithmic-statistical disconnect, with an eye to bridging the gap between these two very different approaches. A concept that lies at the heart of this disconnect is that of statistical regularization, a notion that has to do with how robust is the output of an algorithm to the noise properties of the input data. Although it is nearly completely absent from computer science, which historically has taken the input data as given and modeled algorithms discretely, regularization in one form or another is central to nearly every application domain that applies algorithms to noisy data. By using several case studies, I will illustrate, both theoretically and empirically, the nonobvious fact that approximate computation, in and of itself, can implicitly lead to statistical regularization. This and other recent work suggests that, by exploiting in a more principled way the statistical properties implicit in worst-case algorithms, one can in many cases satisfy the bicriteria of having algorithms that are scalable to very large-scale databases and that also have good inferential or predictive properties.Comment: To appear in the Proceedings of the 2012 ACM Symposium on Principles of Database Systems (PODS 2012