TLib: A Flexible C++ Tensor Framework for Numerical Tensor Calculus

Numerical tensor calculus comprise basic tensor operations such as the entrywise addition and contraction of higher-order tensors. We present, TLib, flexible tensor framework with generic tensor functions and tensor classes that assists users to implement generic and flexible tensor algorithms in C++. The number of dimensions, the extents of the dimensions of the tensors and the contraction modes of the tensor operations can be runtime variable. Our framework provides tensor classes that simplify the management of multidimensional data and utilization of tensor operations using object-oriented and generic programming techniques. Additional stream classes help the user to verify and compare of numerical results with MATLAB. Tensor operations are implemented with generic tensor functions and in terms of multidimensional iterator types only, decoupling data storage representation and computation. The user can combine tensor functions with different tensor types and extend the framework without further modification of the classes or functions. We discuss the design and implementation of the framework and demonstrate its usage with examples that have been discussed in the literature.Comment: 29 page

A compression scheme for radio data in high performance computing

We present a procedure for efficiently compressing astronomical radio data for high performance applications. Integrated, post-correlation data are first passed through a nearly lossless rounding step which compares the precision of the data to a generalized and calibration-independent form of the radiometer equation. This allows the precision of the data to be reduced in a way that has an insignificant impact on the data. The newly developed Bitshuffle lossless compression algorithm is subsequently applied. When the algorithm is used in conjunction with the HDF5 library and data format, data produced by the CHIME Pathfinder telescope is compressed to 28% of its original size and decompression throughputs in excess of 1 GB/s are obtained on a single core.Comment: 12 pages, 3 figures, 2 tables. As published in Astronomy and Computing special issue on "The future of astronomical data formats

Homotopic Group ICA for Multi-Subject Brain Imaging Data

Independent Component Analysis (ICA) is a computational technique for revealing latent factors that underlie sets of measurements or signals. It has become a standard technique in functional neuroimaging. In functional neuroimaging, so called group ICA (gICA) seeks to identify and quantify networks of correlated regions across subjects. This paper reports on the development of a new group ICA approach, Homotopic Group ICA (H-gICA), for blind source separation of resting state functional magnetic resonance imaging (fMRI) data. Resting state brain functional homotopy is the similarity of spontaneous fluctuations between bilaterally symmetrically opposing regions (i.e. those symmetric with respect to the mid-sagittal plane) (Zuo et al., 2010). The approach we proposed improves network estimates by leveraging this known brain functional homotopy. H-gICA increases the potential for network discovery, effectively by averaging information across hemispheres. It is theoretically proven to be identical to standard group ICA when the true sources are both perfectly homotopic and noise-free, while simulation studies and data explorations demonstrate its benefits in the presence of noise. Moreover, compared to commonly applied group ICA algorithms, the structure of the H-gICA input data leads to significant improvement in computational efficiency. A simulation study comfirms its effectiveness in homotopic, non-homotopic and mixed settings, as well as on the landmark ADHD-200 dataset. From a relatively small subset of data, several brain networks were found including: the visual, the default mode and auditory networks, as well as others. These were shown to be more contiguous and clearly delineated than the corresponding ordinary group ICA. Finally, in addition to improving network estimation, H-gICA facilitates the investigation of functional homotopy via ICA-based networks.Comment: 35 pages, 13 figure

Efficient cache oblivious algorithms for randomized divide-and-conquer on the multicore model

In this paper we present randomized algorithms for sorting and convex hull that achieves optimal performance (for speed-up and cache misses) on the multicore model with private cache model. Our algorithms are cache oblivious and generalize the randomized divide and conquer strategy given by Reischuk and Reif and Sen. Although the approach yielded optimal speed-up in the PRAM model, we require additional techniques to optimize cache-misses in an oblivious setting. Under a mild assumption on input and number of processors our algorithm will have optimal time and cache misses with high probability. Although similar results have been obtained recently for sorting, we feel that our approach is simpler and general and we apply it to obtain an optimal parallel algorithm for 3D convex hulls with similar bounds. We also present a simple randomized processor allocation technique without the explicit knowledge of the number of processors that is likely to find additional applications in resource oblivious environments

Submatrix Maximum Queries in Monge Matrices are Equivalent to Predecessor Search

We present an optimal data structure for submatrix maximum queries in n x n Monge matrices. Our result is a two-way reduction showing that the problem is equivalent to the classical predecessor problem in a universe of polynomial size. This gives a data structure of O(n) space that answers submatrix maximum queries in O(loglogn) time. It also gives a matching lower bound, showing that O(loglogn) query-time is optimal for any data structure of size O(n polylog(n)). Our result concludes a line of improvements that started in SODA'12 with O(log^2 n) query-time and continued in ICALP'14 with O(log n) query-time. Finally, we show that partial Monge matrices can be handled in the same bounds as full Monge matrices. In both previous results, partial Monge matrices incurred additional inverse-Ackerman factors

SoftNull: Many-Antenna Full-Duplex Wireless via Digital Beamforming

In this paper, we present and study a digital-controlled method, called SoftNull, to enable full-duplex in many-antenna systems. Unlike most designs that rely on analog cancelers to suppress self-interference, SoftNull relies on digital transmit beamforming to reduce self-interference. SoftNull does not attempt to perfectly null self-interference, but instead seeks to reduce self-interference sufficiently to prevent swamping the receiver's dynamic range. Residual self-interference is then cancelled digitally by the receiver. We evaluate the performance of SoftNull using measurements from a 72-element antenna array in both indoor and outdoor environments. We find that SoftNull can significantly outperform half-duplex for small cells operating in the many-antenna regime, where the number of antennas is many more than the number of users served simultaneously

Resolving Weak Sources within a Dense Array using a Network Approach

A non-parametric technique to identify weak sources within dense sensor arrays is developed using a network approach. No knowledge about the propagation medium is needed except that signal strengths decay to insignificant levels within a scale that is shorter than the aperture. We then reinterpret the spatial covariance matrix of a wave field as a matrix whose support is a connectivity matrix of a network of vertices (sensors) connected into communities. These communities correspond to sensor clusters associated with individual sources. We estimate the support of the covariance matrix from limited-time data using a robust hypothesis test combined with a physical distance criterion. The latter ensures sufficient network sparsity to prevent vertex communities from forming by chance. We verify the approach on simulated data and quantify its reliability. The method is then applied to data from a dense 5200 element geophone array that blanketed 7$\times$10 km of the city of Long Beach (CA). The analysis exposes a helicopter traversing the array, oil production facilities, and reveals that low-frequency events tend to occur near roads

Deep Learning for Bug-Localization in Student Programs

Providing feedback is an integral part of teaching. Most open online courses on programming make use of automated grading systems to support programming assignments and give real-time feedback. These systems usually rely on test results to quantify the programs' functional correctness. They return failing tests to the students as feedback. However, students may find it difficult to debug their programs if they receive no hints about where the bug is and how to fix it. In this work, we present the first deep learning based technique that can localize bugs in a faulty program w.r.t. a failing test, without even running the program. At the heart of our technique is a novel tree convolutional neural network which is trained to predict whether a program passes or fails a given test. To localize the bugs, we analyze the trained network using a state-of-the-art neural prediction attribution technique and see which lines of the programs make it predict the test outcomes. Our experiments show that the proposed technique is generally more accurate than two state-of-the-art program-spectrum based and one syntactic difference based bug-localization baselines

A Unified Approach to Sparse Signal Processing

A unified view of sparse signal processing is presented in tutorial form by bringing together various fields. For each of these fields, various algorithms and techniques, which have been developed to leverage sparsity, are described succinctly. The common benefits of significant reduction in sampling rate and processing manipulations are revealed. The key applications of sparse signal processing are sampling, coding, spectral estimation, array processing, component analysis, and multipath channel estimation. In terms of reconstruction algorithms, linkages are made with random sampling, compressed sensing and rate of innovation. The redundancy introduced by channel coding in finite/real Galois fields is then related to sampling with similar reconstruction algorithms. The methods of Prony, Pisarenko, and MUSIC are next discussed for sparse frequency domain representations. Specifically, the relations of the approach of Prony to an annihilating filter and Error Locator Polynomials in coding are emphasized; the Pisarenko and MUSIC methods are further improvements of the Prony method. Such spectral estimation methods is then related to multi-source location and DOA estimation in array processing. The notions of sparse array beamforming and sparse sensor networks are also introduced. Sparsity in unobservable source signals is also shown to facilitate source separation in SCA; the algorithms developed in this area are also widely used in compressed sensing. Finally, the multipath channel estimation problem is shown to have a sparse formulation; algorithms similar to sampling and coding are used to estimate OFDM channels.Comment: 43 pages, 40 figures, 15 table

A Unified Framework for Identifiability Analysis in Bilinear Inverse Problems with Applications to Subspace and Sparsity Models

Bilinear inverse problems (BIPs), the resolution of two vectors given their image under a bilinear mapping, arise in many applications. Without further constraints, BIPs are usually ill-posed. In practice, properties of natural signals are exploited to solve BIPs. For example, subspace constraints or sparsity constraints are imposed to reduce the search space. These approaches have shown some success in practice. However, there are few results on uniqueness in BIPs. For most BIPs, the fundamental question of under what condition the problem admits a unique solution, is yet to be answered. For example, blind gain and phase calibration (BGPC) is a structured bilinear inverse problem, which arises in many applications, including inverse rendering in computational relighting (albedo estimation with unknown lighting), blind phase and gain calibration in sensor array processing, and multichannel blind deconvolution (MBD). It is interesting to study the uniqueness of such problems. In this paper, we define identifiability of a BIP up to a group of transformations. We derive necessary and sufficient conditions for such identifiability, i.e., the conditions under which the solutions can be uniquely determined up to the transformation group. Applying these results to BGPC, we derive sufficient conditions for unique recovery under several scenarios, including subspace, joint sparsity, and sparsity models. For BGPC with joint sparsity or sparsity constraints, we develop a procedure to compute the relevant transformation groups. We also give necessary conditions in the form of tight lower bounds on sample complexities, and demonstrate the tightness of these bounds by numerical experiments. The results for BGPC not only demonstrate the application of the proposed general framework for identifiability analysis, but are also of interest in their own right.Comment: 40 pages, 3 figure