An Efficient Quadrature Sequence and Sparsifying Methodology for Mean-Field Variational Inference

This work proposes a quasirandom sequence of quadratures for high-dimensional mean-field variational inference and a related sparsifying methodology. Each iterate of the sequence contains two evaluations points that combine to correctly integrate all univariate quadratic functions, as well as univariate cubics if the mean-field factors are symmetric. More importantly, averaging results over short subsequences achieves periodic exactness on a much larger space of multivariate polynomials of quadratic total degree. This framework is devised by first considering stochastic blocked mean-field quadratures, which may be useful in other contexts. By replacing pseudorandom sequences with quasirandom sequences, over half of all multivariate quadratic basis functions integrate exactly with only 4 function evaluations, and the exactness dimension increases for longer subsequences. Analysis shows how these efficient integrals characterize the dominant log-posterior contributions to mean-field variational approximations, including diagonal Hessian approximations, to support a robust sparsifying methodology in deep learning algorithms. A numerical demonstration of this approach on a simple Convolutional Neural Network for MNIST retains high test accuracy, 96.9%, while training over 98.9% of parameters to zero in only 10 epochs, bearing potential to reduce both storage and energy requirements for deep learning models

CABE : a cloud-based acoustic beamforming emulator for FPGA-based sound source localization

Microphone arrays are gaining in popularity thanks to the availability of low-cost microphones. Applications including sonar, binaural hearing aid devices, acoustic indoor localization techniques and speech recognition are proposed by several research groups and companies. In most of the available implementations, the microphones utilized are assumed to offer an ideal response in a given frequency domain. Several toolboxes and software can be used to obtain a theoretical response of a microphone array with a given beamforming algorithm. However, a tool facilitating the design of a microphone array taking into account the non-ideal characteristics could not be found. Moreover, generating packages facilitating the implementation on Field Programmable Gate Arrays has, to our knowledge, not been carried out yet. Visualizing the responses in 2D and 3D also poses an engineering challenge. To alleviate these shortcomings, a scalable Cloud-based Acoustic Beamforming Emulator (CABE) is proposed. The non-ideal characteristics of microphones are considered during the computations and results are validated with acoustic data captured from microphones. It is also possible to generate hardware description language packages containing delay tables facilitating the implementation of Delay-and-Sum beamformers in embedded hardware. Truncation error analysis can also be carried out for fixed-point signal processing. The effects of disabling a given group of microphones within the microphone array can also be calculated. Results and packages can be visualized with a dedicated client application. Users can create and configure several parameters of an emulation, including sound source placement, the shape of the microphone array and the required signal processing flow. Depending on the user configuration, 2D and 3D graphs showing the beamforming results, waterfall diagrams and performance metrics can be generated by the client application. The emulations are also validated with captured data from existing microphone arrays.</jats:p

A supervised learning approach involving active subspaces for an efficient genetic algorithm in high-dimensional optimization problems

In this work, we present an extension of genetic algorithm (GA) which exploits the supervised learning technique called active subspaces (AS) to evolve the individuals on a lower-dimensional space. In many cases, GA requires in fact more function evaluations than other optimization methods to converge to the global optimum. Thus, complex and high-dimensional functions can end up extremely demanding (from the computational point of view) to be optimized with the standard algorithm. To address this issue, we propose to linearly map the input parameter space of the original function onto its AS before the evolution, performing the mutation and mate processes in a lower-dimensional space. In this contribution, we describe the novel method called ASGA, presenting differences and similarities with the standard GA method. We test the proposed method over n-dimensional benchmark functions-Rosenbrock, Ackley, Bohachevsky, Rastrigin, Schaffer N. 7, and Zakharov-and finally we apply it to an aeronautical shape optimization problem

A Moment-Matching Approach to Testable Learning and a New Characterization of Rademacher Complexity

A remarkable recent paper by Rubinfeld and Vasilyan (2022) initiated the study of \emph{testable learning}, where the goal is to replace hard-to-verify distributional assumptions (such as Gaussianity) with efficiently testable ones and to require that the learner succeed whenever the unknown distribution passes the corresponding test. In this model, they gave an efficient algorithm for learning halfspaces under testable assumptions that are provably satisfied by Gaussians. In this paper we give a powerful new approach for developing algorithms for testable learning using tools from moment matching and metric distances in probability. We obtain efficient testable learners for any concept class that admits low-degree \emph{sandwiching polynomials}, capturing most important examples for which we have ordinary agnostic learners. We recover the results of Rubinfeld and Vasilyan as a corollary of our techniques while achieving improved, near-optimal sample complexity bounds for a broad range of concept classes and distributions. Surprisingly, we show that the information-theoretic sample complexity of testable learning is tightly characterized by the Rademacher complexity of the concept class, one of the most well-studied measures in statistical learning theory. In particular, uniform convergence is necessary and sufficient for testable learning. This leads to a fundamental separation from (ordinary) distribution-specific agnostic learning, where uniform convergence is sufficient but not necessary.Comment: 34 page

Development of a quantitative health index and diagnostic method for efficient asset management of power transformers

Power transformers play a very important role in electrical power networks and are frequently operated longer than their expected design life. Therefore, to ensure their best operating performance in a transmission network, the fault condition of each transformer must be assessed regularly. For an accurate fault diagnosis, it is important to have maximum information about an individual transformer based on unbiased measurements. This can best be achieved using artificial intelligence (AI) that can systematically analyse the complex features of diagnostic measurements. Clustering techniques are a form of AI that is particularly well suited to fault diagnosis. To provide an assessment of transformers, a hybrid k-means algorithm, and probabilistic Parzen window estimation are used in this research. The clusters they form are representative of a single or multiple fault categories. The proposed technique computes the maximum probability of transformers in each cluster to determine their fault categories. The main focus of this research is to determine a quantitative health index (HI) to characterize the operating condition of transformers. Condition assessment tries to detect incipient faults before they become too serious, which requires a sensitive and quantified approach. Therefore, the HI needs to come from a proportionate system that can estimate health condition of transformers over time. To quantify this condition, the General Regression Neural Network (GRNN), a type of AI, has been chosen in this research. The GRNN works well with small sets of training data and avoids the needs to estimate large sets of model parameters, following a largely non-parametric approach. The methodology used here regards transformers as a collection of subsystems and summarizes their individual condition into a quantified HI based on the existing agreed benchmarks drawn from IEEE and CIGRE standards. To better calibrate the HI, it may be mapped to a failure probability estimate for each transformer over the coming year. Experimental results of the research show that the proposed methods are more effective than previously published approaches when diagnosing critical faults. Moreover, this novel HI approach can provide a comprehensive assessment of transformers based on the actual condition of their individual subsystems

Development of a quantitative health index and diagnostic method for efficient asset management of power transformers

Power transformers play a very important role in electrical power networks and are frequently operated longer than their expected design life. Therefore, to ensure their best operating performance in a transmission network, the fault condition of each transformer must be assessed regularly. For an accurate fault diagnosis, it is important to have maximum information about an individual transformer based on unbiased measurements. This can best be achieved using artificial intelligence (AI) that can systematically analyse the complex features of diagnostic measurements. Clustering techniques are a form of AI that is particularly well suited to fault diagnosis. To provide an assessment of transformers, a hybrid k-means algorithm, and probabilistic Parzen window estimation are used in this research. The clusters they form are representative of a single or multiple fault categories. The proposed technique computes the maximum probability of transformers in each cluster to determine their fault categories. The main focus of this research is to determine a quantitative health index (HI) to characterize the operating condition of transformers. Condition assessment tries to detect incipient faults before they become too serious, which requires a sensitive and quantified approach. Therefore, the HI needs to come from a proportionate system that can estimate health condition of transformers over time. To quantify this condition, the General Regression Neural Network (GRNN), a type of AI, has been chosen in this research. The GRNN works well with small sets of training data and avoids the needs to estimate large sets of model parameters, following a largely non-parametric approach. The methodology used here regards transformers as a collection of subsystems and summarizes their individual condition into a quantified HI based on the existing agreed benchmarks drawn from IEEE and CIGRE standards. To better calibrate the HI, it may be mapped to a failure probability estimate for each transformer over the coming year. Experimental results of the research show that the proposed methods are more effective than previously published approaches when diagnosing critical faults. Moreover, this novel HI approach can provide a comprehensive assessment of transformers based on the actual condition of their individual subsystems

Bayesian Quadrature for Multiple Related Integrals

Bayesian probabilistic numerical methods are a set of tools providing posterior distributions on the output of numerical methods. The use of these methods is usually motivated by the fact that they can represent our uncertainty due to incomplete/finite information about the continuous mathematical problem being approximated. In this paper, we demonstrate that this paradigm can provide additional advantages, such as the possibility of transferring information between several numerical methods. This allows users to represent uncertainty in a more faithful manner and, as a by-product, provide increased numerical efficiency. We propose the first such numerical method by extending the well-known Bayesian quadrature algorithm to the case where we are interested in computing the integral of several related functions. We then prove convergence rates for the method in the well-specified and misspecified cases, and demonstrate its efficiency in the context of multi-fidelity models for complex engineering systems and a problem of global illumination in computer graphics.Comment: Proceedings of the 35th International Conference on Machine Learning (ICML), PMLR 80:5369-5378, 201

Optimal experimental design for parameter identification and model selection

Magdeburg, Univ., Fak. für Elektrotechnik und Informationstechnik, Diss., 2014René Schenkendor