Parallel Deterministic and Stochastic Global Minimization of Functions with Very Many Minima

The optimization of three problems with high dimensionality and many local minima are investigated under five different optimization algorithms: DIRECT, simulated annealing, Spall’s SPSA algorithm, the KNITRO package, and QNSTOP, a new algorithm developed at Indiana University

Bayesian inference for inverse problems

Traditionally, the MaxEnt workshops start by a tutorial day. This paper summarizes my talk during 2001'th workshop at John Hopkins University. The main idea in this talk is to show how the Bayesian inference can naturally give us all the necessary tools we need to solve real inverse problems: starting by simple inversion where we assume to know exactly the forward model and all the input model parameters up to more realistic advanced problems of myopic or blind inversion where we may be uncertain about the forward model and we may have noisy data. Starting by an introduction to inverse problems through a few examples and explaining their ill posedness nature, I briefly presented the main classical deterministic methods such as data matching and classical regularization methods to show their limitations. I then presented the main classical probabilistic methods based on likelihood, information theory and maximum entropy and the Bayesian inference framework for such problems. I show that the Bayesian framework, not only generalizes all these methods, but also gives us natural tools, for example, for inferring the uncertainty of the computed solutions, for the estimation of the hyperparameters or for handling myopic or blind inversion problems. Finally, through a deconvolution problem example, I presented a few state of the art methods based on Bayesian inference particularly designed for some of the mass spectrometry data processing problems.Comment: Presented at MaxEnt01. To appear in Bayesian Inference and Maximum Entropy Methods, B. Fry (Ed.), AIP Proceedings. 20pages, 13 Postscript figure

The information bottleneck method

We define the relevant information in a signal $x\in X$ as being the information that this signal provides about another signal y\in \Y. Examples include the information that face images provide about the names of the people portrayed, or the information that speech sounds provide about the words spoken. Understanding the signal $x$ requires more than just predicting $y$, it also requires specifying which features of \X play a role in the prediction. We formalize this problem as that of finding a short code for \X that preserves the maximum information about \Y. That is, we squeeze the information that \X provides about \Y through a `bottleneck' formed by a limited set of codewords \tX. This constrained optimization problem can be seen as a generalization of rate distortion theory in which the distortion measure d(x,\x) emerges from the joint statistics of \X and \Y. This approach yields an exact set of self consistent equations for the coding rules X \to \tX and \tX \to \Y. Solutions to these equations can be found by a convergent re-estimation method that generalizes the Blahut-Arimoto algorithm. Our variational principle provides a surprisingly rich framework for discussing a variety of problems in signal processing and learning, as will be described in detail elsewhere

Data-Driven Computing in Dynamics

We formulate extensions to Data Driven Computing for both distance minimizing and entropy maximizing schemes to incorporate time integration. Previous works focused on formulating both types of solvers in the presence of static equilibrium constraints. Here formulations assign data points a variable relevance depending on distance to the solution and on maximum-entropy weighting, with distance minimizing schemes discussed as a special case. The resulting schemes consist of the minimization of a suitably-defined free energy over phase space subject to compatibility and a time-discretized momentum conservation constraint. The present selected numerical tests that establish the convergence properties of both types of Data Driven solvers and solutions.Comment: arXiv admin note: substantial text overlap with arXiv:1702.0157

Digital Signal Processing Research Program

Contains table of contents for Section 2, an introduction and reports on fourteen research projects.U.S. Navy - Office of Naval Research Grant N00014-91-J-1628Defense Advanced Research Projects Agency/U.S. Navy - Office of Naval Research Grant N00014-89-J-1489MIT - Woods Hole Oceanographic Institution Joint ProgramLockheed Sanders, Inc./U.S. Navy Office of Naval Research Contract N00014-91-C-0125U.S. Air Force - Office of Scientific Research Grant AFOSR-91-0034U.S. Navy - Office of Naval Research Grant N00014-91-J-1628AT&T Laboratories Doctoral Support ProgramNational Science Foundation Fellowshi