Fourier Policy Gradients

We propose a new way of deriving policy gradient updates for reinforcement learning. Our technique, based on Fourier analysis, recasts integrals that arise with expected policy gradients as convolutions and turns them into multiplications. The obtained analytical solutions allow us to capture the low variance benefits of EPG in a broad range of settings. For the critic, we treat trigonometric and radial basis functions, two function families with the universal approximation property. The choice of policy can be almost arbitrary, including mixtures or hybrid continuous-discrete probability distributions. Moreover, we derive a general family of sample-based estimators for stochastic policy gradients, which unifies existing results on sample-based approximation. We believe that this technique has the potential to shape the next generation of policy gradient approaches, powered by analytical results

Increasing the Numeric Expressiveness of the Planning Domain Definition Language

The technology of artificial intelligence (AI) planning is being adopted across many different disciplines. This has resulted in the wider use of the Planning Domain Definition Language (PDDL), where it is being used to model planning problems of different natures. One such area where AI planning is particularly attractive is engineering, where the optimisation problems are mathematically rich. The example used throughout this paper is the optimisation (minimisation) of machine tool measurement uncertainty. This planning problem highlights the limits of PDDL's numerical expressiveness in the absence of the square root function. A workaround method using the Babylonian algorithm is then evaluated before the extension of PDDL to include more mathematics functions is discussed

Stochastic collocation on unstructured multivariate meshes

Collocation has become a standard tool for approximation of parameterized systems in the uncertainty quantification (UQ) community. Techniques for least-squares regularization, compressive sampling recovery, and interpolatory reconstruction are becoming standard tools used in a variety of applications. Selection of a collocation mesh is frequently a challenge, but methods that construct geometrically "unstructured" collocation meshes have shown great potential due to attractive theoretical properties and direct, simple generation and implementation. We investigate properties of these meshes, presenting stability and accuracy results that can be used as guides for generating stochastic collocation grids in multiple dimensions.Comment: 29 pages, 6 figure

Low complexity MIMO detection algorithms and implementations

University of Minnesota Ph.D. dissertation. December 2014. Major: Electrical Engineering. Advisor: Gerald E. Sobelman. 1 computer file (PDF); ix, 111 pages.MIMO techniques use multiple antennas at both the transmitter and receiver sides to achieve diversity gain, multiplexing gain, or both. One of the key challenges in exploiting the potential of MIMO systems is to design high-throughput, low-complexity detection algorithms while achieving near-optimal performance. In this thesis, we design and optimize algorithms for MIMO detection and investigate the associated performance and FPGA implementation aspects.First, we study and optimize a detection algorithm developed by Shabany and Gulak for a K-Best based high throughput and low energy hard output MIMO detection and expand it to the complex domain. The new method uses simple lookup tables, and it is fully scalable for a wide range of K-values and constellation sizes. This technique reduces the computational complexity, without sacrificing performance and the complexity scales only sub-linearly with the constellation size. Second, we apply the bidirectional technique to trellis search and propose a high performance soft output bidirectional path preserving trellis search (PPTS) detector for MIMO systems. The comparative error analysis between single direction and bidirectional PPTS detectors is given. We demonstrate that the bidirectional PPTS detector can minimize the detection error. Next, we design a novel bidirectional processing algorithm for soft-output MIMO systems. It combines features from several types of fixed complexity tree search procedures. The proposed approach achieves a higher performance than previously proposed algorithms and has a comparable computational cost. Moreover, its parallel nature and fixed throughput characteristics make it attractive for very large scale integration (VLSI) implementation.Following that, we present a novel low-complexity hard output MIMO detection algorithm for LTE and WiFi applications. We provide a well-defined tradeoff between computational complexity and performance. The proposed algorithm uses a much smaller number of Euclidean distance (ED) calculations while attaining only a 0.5dB loss compared to maximum likelihood detection (MLD). A 3x3 MIMO system with a 16QAM detector architecture is designed, and the latency and hardware costs are estimated.Finally, we present a stochastic computing implementation of trigonometric and hyperbolic functions which can be used for QR decomposition and other wireless communications and signal processing applications

Finite element approximation of high-dimensional transport-dominated diffusion problems

High-dimensional partial differential equations with nonnegative characteristic form arise in numerous mathematical models in science. In problems of this kind, the computational challenge of beating the exponential growth of complexity as a function of dimension is exacerbated by the fact that the problem may be transport-dominated. We develop the analysis of stabilised sparse finite element methods for such high-dimensional, non-self-adjoint and possibly degenerate partial differential equations.\ud \ud (Presented as an invited lecture under the title "Computational multiscale modelling: Fokker-Planck equations and their numerical analysis" at the Foundations of Computational Mathematics conference in Santander, Spain, 30 June - 9 July, 2005.