High dimensional inference: structured sparse models and non-linear measurement channels

Thesis (Ph.D.)--Boston UniversityHigh dimensional inference is motivated by many real life problems such as medical diagnosis, security, and marketing. In statistical inference problems, n data samples are collected where each sample contains p attributes. High dimensional inference deals with problems in which the number of parameters, p, is larger than the sample size, n. To hope for any consistent result within high dimensional framework, data is assumed to lie on a low dimensional manifold. This implies that only k « p parameters are required to characterize p feature variables. One way to impose such a low dimensional structure is a regularization based approach. In this approach, statistical inference problem is mapped to an optimization problem in which a regularizer term penalizes the deviation of the model from a specific structure. The choice of appropriate penalizing functions is often challenging. We explore three major problems that arise in the context of this approach. First, we probe the reconstruction problem under sparse Poisson models. We are motivated by applications in explosive identification, and online marketing where the observations are the counts of a recurring event. We study the amplitude effect which distinguishes our problem from a conventional linear regression least squares problem. Motivated by applications in decentralized sensor networks and distributed multi-task learning, we study the effect of decentralization on high dimensional inference. Finally, we provide a general framework to study the impact of multiple structured models on performance of regularization based reconstruction methods. For each of the afore- mentioned scenarios, we propose an equivalent optimization problem and specify the conditions under which the optimization problem can be solved. Moreover, we mathematically analyze the performance of such recovery method in terms of reconstruction error, prediction error, probability of successful recovery, and sample complexity

The Complexity of Three-Way Statistical Tables

Multi-way tables with specified marginals arise in a variety of applications in statistics and operations research. We provide a comprehensive complexity classification of three fundamental computational problems on tables: existence, counting and entry-security. One major outcome of our work is that each of the following problems is intractable already for "slim" 3-tables, with constant and smallest possible number 3 of rows: (1) deciding existence of 3-tables with given consistent 2-marginals; (2) counting all 3-tables with given 2-marginals; (3) finding whether an integer value is attained in entry (i,j,k) by at least one of the 3-tables satisfying given (feasible) 2-marginals. This implies that a characterization of feasible marginals for such slim tables, sought by much recent research, is unlikely to exist. Another important consequence of our study is a systematic efficient way of embedding the set of 3-tables satisfying any given 1-marginals and entry upper bounds in a set of slim 3-tables satisfying suitable 2-marginals with no entry bounds. This provides a valuable tool for studying multi-index transportation problems and multi-index transportation polytopes

A Modern Framework for Measuring Poverty and Basic Economic Security

This report details how the dominant framework for understanding and measuring poverty in the United States has become a conservative one. The current U.S. approach to measuring poverty views poverty only in terms of having an extremely low level of annual income, and utilizes poverty thresholds that are adjusted only for inflation rather than for changes in overall living standards. As a result, the official poverty measure has effectively defined deprivation down over the last four decades, moving it further and further away from mainstream living standards over time, as well as from majority public opinion of the minimum amount needed to "get along" at a basic level. A new Supplemental Income Poverty Measure (SIPM) proposed by the Obama administration makes some important improvements to the current poverty measure. However, the SIPM remains a conservative approach that appears likely to lock in the poverty line at an extremely low level. This report proposes a new framework for measuring poverty and basic economic security in the United States. Instead of being limited to the "extremely-low-income-only" approach the current poverty line and administration's proposed Supplemental Income Poverty Measure (SIPM) represent, this framework should utilize measures of low income and other forms of economic hardship related to low income

Yet Another Pseudorandom Number Generator

We propose a novel pseudorandom number generator based on R\"ossler attractor and bent Boolean function. We estimated the output bits properties by number of statistical tests. The results of the cryptanalysis show that the new pseudorandom number generation scheme provides a high level of data security.Comment: 5 pages, 7 figures; to be published in International Journal of Electronics and Telecommunications, vol.63, no.

Training Process Reduction Based On Potential Weights Linear Analysis To Accelarate Back Propagation Network

Learning is the important property of Back Propagation Network (BPN) and finding the suitable weights and thresholds during training in order to improve training time as well as achieve high accuracy. Currently, data pre-processing such as dimension reduction input values and pre-training are the contributing factors in developing efficient techniques for reducing training time with high accuracy and initialization of the weights is the important issue which is random and creates paradox, and leads to low accuracy with high training time. One good data preprocessing technique for accelerating BPN classification is dimension reduction technique but it has problem of missing data. In this paper, we study current pre-training techniques and new preprocessing technique called Potential Weight Linear Analysis (PWLA) which combines normalization, dimension reduction input values and pre-training. In PWLA, the first data preprocessing is performed for generating normalized input values and then applying them by pre-training technique in order to obtain the potential weights. After these phases, dimension of input values matrix will be reduced by using real potential weights. For experiment results XOR problem and three datasets, which are SPECT Heart, SPECTF Heart and Liver disorders (BUPA) will be evaluated. Our results, however, will show that the new technique of PWLA will change BPN to new Supervised Multi Layer Feed Forward Neural Network (SMFFNN) model with high accuracy in one epoch without training cycle. Also PWLA will be able to have power of non linear supervised and unsupervised dimension reduction property for applying by other supervised multi layer feed forward neural network model in future work.Comment: 11 pages IEEE format, International Journal of Computer Science and Information Security, IJCSIS 2009, ISSN 1947 5500, Impact factor 0.42