Compressed sensing and robust recovery of low rank matrices

In this paper, we focus on compressed sensing and recovery schemes for low-rank matrices, asking under what conditions a low-rank matrix can be sensed and recovered from incomplete, inaccurate, and noisy observations. We consider three schemes, one based on a certain Restricted Isometry Property and two based on directly sensing the row and column space of the matrix. We study their properties in terms of exact recovery in the ideal case, and robustness issues for approximately low-rank matrices and for noisy measurements

Asynchronous Training of Word Embeddings for Large Text Corpora

Word embeddings are a powerful approach for analyzing language and have been widely popular in numerous tasks in information retrieval and text mining. Training embeddings over huge corpora is computationally expensive because the input is typically sequentially processed and parameters are synchronously updated. Distributed architectures for asynchronous training that have been proposed either focus on scaling vocabulary sizes and dimensionality or suffer from expensive synchronization latencies. In this paper, we propose a scalable approach to train word embeddings by partitioning the input space instead in order to scale to massive text corpora while not sacrificing the performance of the embeddings. Our training procedure does not involve any parameter synchronization except a final sub-model merge phase that typically executes in a few minutes. Our distributed training scales seamlessly to large corpus sizes and we get comparable and sometimes even up to 45% performance improvement in a variety of NLP benchmarks using models trained by our distributed procedure which requires $1/10$ of the time taken by the baseline approach. Finally we also show that we are robust to missing words in sub-models and are able to effectively reconstruct word representations.Comment: This paper contains 9 pages and has been accepted in the WSDM201

BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees

The rising volume of datasets has made training machine learning (ML) models a major computational cost in the enterprise. Given the iterative nature of model and parameter tuning, many analysts use a small sample of their entire data during their initial stage of analysis to make quick decisions (e.g., what features or hyperparameters to use) and use the entire dataset only in later stages (i.e., when they have converged to a specific model). This sampling, however, is performed in an ad-hoc fashion. Most practitioners cannot precisely capture the effect of sampling on the quality of their model, and eventually on their decision-making process during the tuning phase. Moreover, without systematic support for sampling operators, many optimizations and reuse opportunities are lost. In this paper, we introduce BlinkML, a system for fast, quality-guaranteed ML training. BlinkML allows users to make error-computation tradeoffs: instead of training a model on their full data (i.e., full model), BlinkML can quickly train an approximate model with quality guarantees using a sample. The quality guarantees ensure that, with high probability, the approximate model makes the same predictions as the full model. BlinkML currently supports any ML model that relies on maximum likelihood estimation (MLE), which includes Generalized Linear Models (e.g., linear regression, logistic regression, max entropy classifier, Poisson regression) as well as PPCA (Probabilistic Principal Component Analysis). Our experiments show that BlinkML can speed up the training of large-scale ML tasks by 6.26x-629x while guaranteeing the same predictions, with 95% probability, as the full model.Comment: 22 pages, SIGMOD 201

Ab initio study of the vapour-liquid critical point of a symmetrical binary fluid mixture

A microscopic approach to the investigation of the behaviour of a symmetrical binary fluid mixture in the vicinity of the vapour-liquid critical point is proposed. It is shown that the problem can be reduced to the calculation of the partition function of a 3D Ising model in an external field. For a square-well symmetrical binary mixture we calculate the parameters of the critical point as functions of the microscopic parameter r measuring the relative strength of interactions between the particles of dissimilar and similar species. The calculations are performed at intermediate ($\lambda=1.5$) and moderately long ($\lambda=2.0$) intermolecular potential ranges. The obtained results agree well with the ones of computer simulations.Comment: 14 pages, Latex2e, 5 eps-figures included, submitted to J.Phys:Cond.Ma

Estimation in high dimensions: a geometric perspective

This tutorial provides an exposition of a flexible geometric framework for high dimensional estimation problems with constraints. The tutorial develops geometric intuition about high dimensional sets, justifies it with some results of asymptotic convex geometry, and demonstrates connections between geometric results and estimation problems. The theory is illustrated with applications to sparse recovery, matrix completion, quantization, linear and logistic regression and generalized linear models.Comment: 56 pages, 9 figures. Multiple minor change

Sequencing chemotherapy and radiotherapy in locoregional advanced breast cancer patients after mastectomy – a retrospective analysis

<p>Abstract</p> <p>Background</p> <p>Combined chemo- and radiotherapy are established in breast cancer treatment. Chemotherapy is recommended prior to radiotherapy but decisive data on the optimal sequence are rare. This retrospective analysis aimed to assess the role of sequencing in patients after mastectomy because of advanced locoregional disease.</p> <p>Methods</p> <p>A total of 212 eligible patients had a stage III breast cancer and had adjuvant chemotherapy and radiotherapy after mastectomy and axillary dissection between 1996 and 2004. According to concerted multi-modality treatment strategies 86 patients were treated sequentially (chemotherapy followed by radiotherapy) (SEQgroup), 70 patients had a sandwich treatment (SW-group) and 56 patients had simultaneous chemoradiation (SIM-group) during that time period. Radiotherapy comprised the thoracic wall and/or regional lymph nodes. The total dose was 45–50.4 Gray. As simultaneous chemoradiation CMF was given in 95.4% of patients while in sequential or sandwich application in 86% and 87.1% of patients an anthracycline-based chemotherapy was given.</p> <p>Results</p> <p>Concerning the parameters nodal involvement, lymphovascular invasion, extracapsular spread and extension of the irradiated region the three treatment groups were significantly imbalanced. The other parameters, e.g. age, pathological tumor stage, grading and receptor status were homogeneously distributed. Looking on those two groups with an equally effective chemotherapy (EC, FEC), the SEQ- and SW-group, the sole imbalance was the extension of LVI (57.1 vs. 25.6%, p < 0.0001).</p> <p>5-year overall- and disease free survival were 53.2%/56%, 38.1%/32% and 64.2%/50%, for the sequential, sandwich and simultaneous regime, respectively, which differed significantly in the univariate analysis (p = 0.04 and p = 0.03, log-rank test). Also the 5-year locoregional or distant recurrence free survival showed no significant differences according to the sequence of chemo- and radiotherapy. In the multivariate analyses the sequence had no independent impact on overall survival (p = 0.2) or disease free survival (p = 0.4). The toxicity, whether acute nor late, showed no significant differences in the three groups. The grade III/IV acute side effects were 3.6%, 0% and 3.5% for the SIM-, SW- and SEQ-group. By tendency the SIM regime had more late side effects.</p> <p>Conclusion</p> <p>No clear advantage can be stated for any radio- and chemotherapy sequence in breast cancer therapy so far. This could be confirmed in our retrospective analysis in high-risk patients after mastectomy. The sequential approach is recommended according to current guidelines considering a lower toxicity.</p

Quantum Tomography via Compressed Sensing: Error Bounds, Sample Complexity, and Efficient Estimators

Intuitively, if a density operator has small rank, then it should be easier to estimate from experimental data, since in this case only a few eigenvectors need to be learned. We prove two complementary results that confirm this intuition. First, we show that a low-rank density matrix can be estimated using fewer copies of the state, i.e., the sample complexity of tomography decreases with the rank. Second, we show that unknown low-rank states can be reconstructed from an incomplete set of measurements, using techniques from compressed sensing and matrix completion. These techniques use simple Pauli measurements, and their output can be certified without making any assumptions about the unknown state. We give a new theoretical analysis of compressed tomography, based on the restricted isometry property (RIP) for low-rank matrices. Using these tools, we obtain near-optimal error bounds, for the realistic situation where the data contains noise due to finite statistics, and the density matrix is full-rank with decaying eigenvalues. We also obtain upper-bounds on the sample complexity of compressed tomography, and almost-matching lower bounds on the sample complexity of any procedure using adaptive sequences of Pauli measurements. Using numerical simulations, we compare the performance of two compressed sensing estimators with standard maximum-likelihood estimation (MLE). We find that, given comparable experimental resources, the compressed sensing estimators consistently produce higher-fidelity state reconstructions than MLE. In addition, the use of an incomplete set of measurements leads to faster classical processing with no loss of accuracy. Finally, we show how to certify the accuracy of a low rank estimate using direct fidelity estimation and we describe a method for compressed quantum process tomography that works for processes with small Kraus rank.Comment: 16 pages, 3 figures. Matlab code included with the source file

Robust Matrix Completion

This paper considers the problem of recovery of a low-rank matrix in the situation when most of its entries are not observed and a fraction of observed entries are corrupted. The observations are noisy realizations of the sum of a low rank matrix, which we wish to recover, with a second matrix having a complementary sparse structure such as element-wise or column-wise sparsity. We analyze a class of estimators obtained by solving a constrained convex optimization problem that combines the nuclear norm and a convex relaxation for a sparse constraint. Our results are obtained for the simultaneous presence of random and deterministic patterns in the sampling scheme. We provide guarantees for recovery of low-rank and sparse components from partial and corrupted observations in the presence of noise and show that the obtained rates of convergence are minimax optimal