Generalized minimizers of convex integral functionals, Bregman distance, Pythagorean identities

Integral functionals based on convex normal integrands are minimized subject to finitely many moment constraints. The integrands are finite on the positive and infinite on the negative numbers, strictly convex but not necessarily differentiable. The minimization is viewed as a primal problem and studied together with a dual one in the framework of convex duality. The effective domain of the value function is described by a conic core, a modification of the earlier concept of convex core. Minimizers and generalized minimizers are explicitly constructed from solutions of modified dual problems, not assuming the primal constraint qualification. A generalized Pythagorean identity is presented using Bregman distance and a correction term for lack of essential smoothness in integrands. Results are applied to minimization of Bregman distances. Existence of a generalized dual solution is established whenever the dual value is finite, assuming the dual constraint qualification. Examples of `irregular' situations are included, pointing to the limitations of generality of certain key results

Geometric Science of Information

Abstract. Integral functionals based on convex normal integrands are minimized subject to finitely many moment constraints. The effective domain of the value function is described by a modification of the concept of convex core. The minimization is viewed as a primal problem and studied together with a dual one in the framework of convex duality. The minimizers and generalized minimizers are explicitly described whenever the primal value is finite, assuming a dual constraint qualification but not the primal constraint qualification. A generalized Pythagorean identity is presented using Bregman distance and a correction term. The problem Proc. Geometric Science of Information 2013, Springer LNCS 8085, 302-307. This contribution addresses minimization of integral functionals of real functions g on a σ-finite measure space (Z, Z, µ), subject to the constraint that the moment vector Z ϕg dµ of g is prescribed. Here, ϕ is a given R d -valued Z-measurable moment mapping. It is assumed throughout that β is any mapping Z × R → (−∞,+∞] such that β(·, t) is Z-measurable for t ∈ R, and β(z, ·), z ∈ Z, is in the class Γ of functions γ on R that are finite and strictly convex for t > 0, equal to +∞ for t < 0, and satisfy γ(0) = lim t↓0 γ(t). In particular, β is a normal integrand whence z → β(z, g(z)) is Z-measurable if g is. If neither the positive nor the negative part of β(z, g(z)) is µ-integrable, the integral in Given a ∈ R d , let Ga denote the class of those nonnegative Z-measurable functions g whose moment vector exists and equals a. By the assumptions on β, the minimization of H β over g with the moment vector equal to a gives rise to the value functio

Convergence of generalized entropy minimizers in sequences of convex problems

Integral functionals based on convex normal integrands are minimized over convex constraint sets. Generalized minimizers exist under a boundedness condition. Sequences of the minimization problems are studied when the constraint sets are nested. The corresponding sequences of generalized minimizers are related to the minimization over limit convex sets. Martingale theorems and moment problems are discussed. © 2016 IEEE

Minimization Problems Based on Relative α\alpha-Entropy I: Forward Projection

Minimization problems with respect to a one-parameter family of generalized relative entropies are studied. These relative entropies, which we term relative $\alpha$-entropies (denoted $\mathscr{I}_{\alpha}$), arise as redundancies under mismatched compression when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative entropies are a generalization of the usual relative entropy (Kullback-Leibler divergence). Just like relative entropy, these relative $\alpha$-entropies behave like squared Euclidean distance and satisfy the Pythagorean property. Minimizers of these relative $\alpha$-entropies on closed and convex sets are shown to exist. Such minimizations generalize the maximum R\'{e}nyi or Tsallis entropy principle. The minimizing probability distribution (termed forward $\mathscr{I}_{\alpha}$-projection) for a linear family is shown to obey a power-law. Other results in connection with statistical inference, namely subspace transitivity and iterated projections, are also established. In a companion paper, a related minimization problem of interest in robust statistics that leads to a reverse $\mathscr{I}_{\alpha}$-projection is studied.Comment: 24 pages; 4 figures; minor change in title; revised version. Accepted for publication in IEEE Transactions on Information Theor

Paradigms of Cognition

An abstract, quantitative theory which connects elements of information —key ingredients in the cognitive proces—is developed. Seemingly unrelated results are thereby unified. As an indication of this, consider results in classical probabilistic information theory involving information projections and so-called Pythagorean inequalities. This has a certain resemblance to classical results in geometry bearing Pythagoras’ name. By appealing to the abstract theory presented here, you have a common point of reference for these results. In fact, the new theory provides a general framework for the treatment of a multitude of global optimization problems across a range of disciplines such as geometry, statistics and statistical physics. Several applications are given, among them an “explanation” of Tsallis entropy is suggested. For this, as well as for the general development of the abstract underlying theory, emphasis is placed on interpretations and associated philosophical considerations. Technically, game theory is the key tool

Robust Identification of Investor Beliefs

This paper develops a new method informed by data and models to recover information about investor beliefs. Our approach uses information embedded in forward-looking asset prices in conjunction with asset pricing models. We step back from presuming rational expectations and entertain potential belief distortions bounded by a statistical measure of discrepancy. Additionally, our method allows for the direct use of sparse survey evidence to make these bounds more informative. Within our framework, market-implied beliefs may differ from those implied by rational expectations due to behavioral/psychological biases of investors, ambiguity aversion, or omitted permanent components to valuation. Formally, we represent evidence about investor beliefs using a novel nonlinear expectation function deduced using model-implied moment conditions and bounds on statistical divergence. We illustrate our method with a prototypical example from macro-finance using asset market data to infer belief restrictions for macroeconomic growth rates

New Directions for Contact Integrators

Contact integrators are a family of geometric numerical schemes which guarantee the conservation of the contact structure. In this work we review the construction of both the variational and Hamiltonian versions of these methods. We illustrate some of the advantages of geometric integration in the dissipative setting by focusing on models inspired by recent studies in celestial mechanics and cosmology.Comment: To appear as Chapter 24 in GSI 2021, Springer LNCS 1282