2,188 research outputs found
Playing with Duality: An Overview of Recent Primal-Dual Approaches for Solving Large-Scale Optimization Problems
Optimization methods are at the core of many problems in signal/image
processing, computer vision, and machine learning. For a long time, it has been
recognized that looking at the dual of an optimization problem may drastically
simplify its solution. Deriving efficient strategies which jointly brings into
play the primal and the dual problems is however a more recent idea which has
generated many important new contributions in the last years. These novel
developments are grounded on recent advances in convex analysis, discrete
optimization, parallel processing, and non-smooth optimization with emphasis on
sparsity issues. In this paper, we aim at presenting the principles of
primal-dual approaches, while giving an overview of numerical methods which
have been proposed in different contexts. We show the benefits which can be
drawn from primal-dual algorithms both for solving large-scale convex
optimization problems and discrete ones, and we provide various application
examples to illustrate their usefulness
Means on discrete product lattices
We investigate the problem of averaging values on lattices, and in particular on discrete product lattices. This problem arises in image processing when several color values given in RGB, HSL, or another coding scheme, need to be combined. We show how the arithmetic mean and the median can be constructed by minimizing appropriate penalties. We also discuss which of them coincide with the Cartesian product of the standard mean and median
Functional Data Analysis in Electronic Commerce Research
This paper describes opportunities and challenges of using functional data
analysis (FDA) for the exploration and analysis of data originating from
electronic commerce (eCommerce). We discuss the special data structures that
arise in the online environment and why FDA is a natural approach for
representing and analyzing such data. The paper reviews several FDA methods and
motivates their usefulness in eCommerce research by providing a glimpse into
new domain insights that they allow. We argue that the wedding of eCommerce
with FDA leads to innovations both in statistical methodology, due to the
challenges and complications that arise in eCommerce data, and in online
research, by being able to ask (and subsequently answer) new research questions
that classical statistical methods are not able to address, and also by
expanding on research questions beyond the ones traditionally asked in the
offline environment. We describe several applications originating from online
transactions which are new to the statistics literature, and point out
statistical challenges accompanied by some solutions. We also discuss some
promising future directions for joint research efforts between researchers in
eCommerce and statistics.Comment: Published at http://dx.doi.org/10.1214/088342306000000132 in the
Statistical Science (http://www.imstat.org/sts/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Saliency Benchmarking Made Easy: Separating Models, Maps and Metrics
Dozens of new models on fixation prediction are published every year and
compared on open benchmarks such as MIT300 and LSUN. However, progress in the
field can be difficult to judge because models are compared using a variety of
inconsistent metrics. Here we show that no single saliency map can perform well
under all metrics. Instead, we propose a principled approach to solve the
benchmarking problem by separating the notions of saliency models, maps and
metrics. Inspired by Bayesian decision theory, we define a saliency model to be
a probabilistic model of fixation density prediction and a saliency map to be a
metric-specific prediction derived from the model density which maximizes the
expected performance on that metric given the model density. We derive these
optimal saliency maps for the most commonly used saliency metrics (AUC, sAUC,
NSS, CC, SIM, KL-Div) and show that they can be computed analytically or
approximated with high precision. We show that this leads to consistent
rankings in all metrics and avoids the penalties of using one saliency map for
all metrics. Our method allows researchers to have their model compete on many
different metrics with state-of-the-art in those metrics: "good" models will
perform well in all metrics.Comment: published at ECCV 201
Inference with Linear Equality and Inequality Constraints Using R: The Package ic.infer
In linear models and multivariate normal situations, prior information in linear inequality form may be encountered, or linear inequality hypotheses may be subjected to statistical tests. R package ic.infer has been developed to support inequality-constrained estimation and testing for such situations. This article gives an overview of the principles underlying inequality-constrained inference that are far less well-known than methods for unconstrained or equality-constrained models, and describes their implementation in the package.
Recommended from our members
Constraint based approaches to interpretable and semi-supervised machine learning
Interpretability and Explainability of machine learning algorithms are becoming increasingly important as Machine Learning (ML) systems get widely applied to domains like clinical healthcare, social media and governance. A related major challenge in deploying ML systems pertains to reliable learning when expert annotation is severely limited. This dissertation prescribes a common framework to address these challenges, based on the use of constraints that can make an ML model more interpretable, lead to novel methods for explaining ML models, or help to learn reliably with limited supervision.
In particular, we focus on the class of latent variable models and develop a general learning framework by constraining realizations of latent variables and/or model parameters. We propose specific constraints that can be used to develop identifiable latent variable models, that in turn learn interpretable outcomes. The proposed framework is first used in Non–negative Matrix Factorization and Probabilistic Graphical Models. For both models, algorithms are proposed to incorporate such constraints with seamless and tractable augmentation of the associated learning and inference procedures. The utility of the proposed methods is demonstrated for our working application domain – identifiable phenotyping using Electronic Health Records (EHRs). Evaluation by domain experts reveals that the proposed models are indeed more clinically relevant (and hence more interpretable) than existing counterparts. The work also demonstrates that while there may be inherent trade–offs between constraining models to encourage interpretability, the quantitative performance of downstream tasks remains competitive.
We then focus on constraint based mechanisms to explain decisions or outcomes of supervised black-box models. We propose an explanation model based on generating examples where the nature of the examples is constrained i.e. they have to be sampled from the underlying data domain. To do so, we train a generative model to characterize the data manifold in a high dimensional ambient space. Constrained sampling then allows us to generate naturalistic examples that lie along the data manifold. We propose ways to summarize model behavior using such constrained examples.
In the last part of the contributions, we argue that heterogeneity of data sources is useful in situations where very little to no supervision is available. This thesis leverages such heterogeneity (via constraints) for two critical but widely different machine learning algorithms. In each case, a novel algorithm in the sub-class of co–regularization is developed to combine information from heterogeneous sources. Co–regularization is a framework of constraining latent variables and/or latent distributions in order to leverage heterogeneity. The proposed algorithms are utilized for clustering, where the intent is to generate a partition or grouping of observed samples, and for Learning to Rank algorithms – used to rank a set of observed samples in order of preference with respect to a specific search query. The proposed methods are evaluated on clustering web documents, social network users, and information retrieval applications for ranking search queries.Electrical and Computer Engineerin
Dynamics of Information Diffusion and Social Sensing
Statistical inference using social sensors is an area that has witnessed
remarkable progress and is relevant in applications including localizing events
for targeted advertising, marketing, localization of natural disasters and
predicting sentiment of investors in financial markets. This chapter presents a
tutorial description of four important aspects of sensing-based information
diffusion in social networks from a communications/signal processing
perspective. First, diffusion models for information exchange in large scale
social networks together with social sensing via social media networks such as
Twitter is considered. Second, Bayesian social learning models and risk averse
social learning is considered with applications in finance and online
reputation systems. Third, the principle of revealed preferences arising in
micro-economics theory is used to parse datasets to determine if social sensors
are utility maximizers and then determine their utility functions. Finally, the
interaction of social sensors with YouTube channel owners is studied using time
series analysis methods. All four topics are explained in the context of actual
experimental datasets from health networks, social media and psychological
experiments. Also, algorithms are given that exploit the above models to infer
underlying events based on social sensing. The overview, insights, models and
algorithms presented in this chapter stem from recent developments in network
science, economics and signal processing. At a deeper level, this chapter
considers mean field dynamics of networks, risk averse Bayesian social learning
filtering and quickest change detection, data incest in decision making over a
directed acyclic graph of social sensors, inverse optimization problems for
utility function estimation (revealed preferences) and statistical modeling of
interacting social sensors in YouTube social networks.Comment: arXiv admin note: text overlap with arXiv:1405.112
Complexity Theory, Game Theory, and Economics: The Barbados Lectures
This document collects the lecture notes from my mini-course "Complexity
Theory, Game Theory, and Economics," taught at the Bellairs Research Institute
of McGill University, Holetown, Barbados, February 19--23, 2017, as the 29th
McGill Invitational Workshop on Computational Complexity.
The goal of this mini-course is twofold: (i) to explain how complexity theory
has helped illuminate several barriers in economics and game theory; and (ii)
to illustrate how game-theoretic questions have led to new and interesting
complexity theory, including recent several breakthroughs. It consists of two
five-lecture sequences: the Solar Lectures, focusing on the communication and
computational complexity of computing equilibria; and the Lunar Lectures,
focusing on applications of complexity theory in game theory and economics. No
background in game theory is assumed.Comment: Revised v2 from December 2019 corrects some errors in and adds some
recent citations to v1 Revised v3 corrects a few typos in v
- …