Data-Driven Estimation in Equilibrium Using Inverse Optimization

Equilibrium modeling is common in a variety of fields such as game theory and transportation science. The inputs for these models, however, are often difficult to estimate, while their outputs, i.e., the equilibria they are meant to describe, are often directly observable. By combining ideas from inverse optimization with the theory of variational inequalities, we develop an efficient, data-driven technique for estimating the parameters of these models from observed equilibria. We use this technique to estimate the utility functions of players in a game from their observed actions and to estimate the congestion function on a road network from traffic count data. A distinguishing feature of our approach is that it supports both parametric and \emph{nonparametric} estimation by leveraging ideas from statistical learning (kernel methods and regularization operators). In computational experiments involving Nash and Wardrop equilibria in a nonparametric setting, we find that a) we effectively estimate the unknown demand or congestion function, respectively, and b) our proposed regularization technique substantially improves the out-of-sample performance of our estimators.Comment: 36 pages, 5 figures Additional theorems for generalization guarantees and statistical analysis adde

Learning Local Feature Aggregation Functions with Backpropagation

This paper introduces a family of local feature aggregation functions and a novel method to estimate their parameters, such that they generate optimal representations for classification (or any task that can be expressed as a cost function minimization problem). To achieve that, we compose the local feature aggregation function with the classifier cost function and we backpropagate the gradient of this cost function in order to update the local feature aggregation function parameters. Experiments on synthetic datasets indicate that our method discovers parameters that model the class-relevant information in addition to the local feature space. Further experiments on a variety of motion and visual descriptors, both on image and video datasets, show that our method outperforms other state-of-the-art local feature aggregation functions, such as Bag of Words, Fisher Vectors and VLAD, by a large margin.Comment: In Proceedings of the 25th European Signal Processing Conference (EUSIPCO 2017

Best-fit quasi-equilibrium ensembles: a general approach to statistical closure of underresolved Hamiltonian dynamics

A new method of deriving reduced models of Hamiltonian dynamical systems is developed using techniques from optimization and statistical estimation. Given a set of resolved variables that define a model reduction, the quasi-equilibrium ensembles associated with the resolved variables are employed as a family of trial probability densities on phase space. The residual that results from submitting these trial densities to the Liouville equation is quantified by an ensemble-averaged cost function related to the information loss rate of the reduction. From an initial nonequilibrium state, the statistical state of the system at any later time is estimated by minimizing the time integral of the cost function over paths of trial densities. Statistical closure of the underresolved dynamics is obtained at the level of the value function, which equals the optimal cost of reduction with respect to the resolved variables, and the evolution of the estimated statistical state is deduced from the Hamilton-Jacobi equation satisfied by the value function. In the near-equilibrium regime, or under a local quadratic approximation in the far-from-equilibrium regime, this best-fit closure is governed by a differential equation for the estimated state vector coupled to a Riccati differential equation for the Hessian matrix of the value function. Since memory effects are not explicitly included in the trial densities, a single adjustable parameter is introduced into the cost function to capture a time-scale ratio between resolved and unresolved motions. Apart from this parameter, the closed equations for the resolved variables are completely determined by the underlying deterministic dynamics

Adaptive asymptotically efficient estimation in heteroscedastic nonparametric regression

The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper, 2007, for estimating an unknown nonparametric regression. %\cite{GaPe1}. We prove that this procedure is asymptotically efficient for a quadratic risk, i.e. the asymptotic quadratic risk for this procedure coincides with the Pinsker constant which gives a sharp lower bound for the quadratic risk over all possible estimate

Monte Carlo optimization of decentralized estimation networks over directed acyclic graphs under communication constraints

Motivated by the vision of sensor networks, we consider decentralized estimation networks over bandwidth–limited communication links, and are particularly interested in the tradeoff between the estimation accuracy and the cost of communications due to, e.g., energy consumption. We employ a class of in–network processing strategies that admits directed acyclic graph representations and yields a tractable Bayesian risk that comprises the cost of communications and estimation error penalty. This perspective captures a broad range of possibilities for processing under network constraints and enables a rigorous design problem in the form of constrained optimization. A similar scheme and the structures exhibited by the solutions have been previously studied in the context of decentralized detection. Under reasonable assumptions, the optimization can be carried out in a message passing fashion. We adopt this framework for estimation, however, the corresponding optimization scheme involves integral operators that cannot be evaluated exactly in general. We develop an approximation framework using Monte Carlo methods and obtain particle representations and approximate computational schemes for both the in–network processing strategies and their optimization. The proposed Monte Carlo optimization procedure operates in a scalable and efficient fashion and, owing to the non-parametric nature, can produce results for any distributions provided that samples can be produced from the marginals. In addition, this approach exhibits graceful degradation of the estimation accuracy asymptotically as the communication becomes more costly, through a parameterized Bayesian risk