210,054 research outputs found

    Regularized Optimal Transport Layers for Generalized Global Pooling Operations

    Full text link
    Global pooling is one of the most significant operations in many machine learning models and tasks, which works for information fusion and structured data (like sets and graphs) representation. However, without solid mathematical fundamentals, its practical implementations often depend on empirical mechanisms and thus lead to sub-optimal, even unsatisfactory performance. In this work, we develop a novel and generalized global pooling framework through the lens of optimal transport. The proposed framework is interpretable from the perspective of expectation-maximization. Essentially, it aims at learning an optimal transport across sample indices and feature dimensions, making the corresponding pooling operation maximize the conditional expectation of input data. We demonstrate that most existing pooling methods are equivalent to solving a regularized optimal transport (ROT) problem with different specializations, and more sophisticated pooling operations can be implemented by hierarchically solving multiple ROT problems. Making the parameters of the ROT problem learnable, we develop a family of regularized optimal transport pooling (ROTP) layers. We implement the ROTP layers as a new kind of deep implicit layer. Their model architectures correspond to different optimization algorithms. We test our ROTP layers in several representative set-level machine learning scenarios, including multi-instance learning (MIL), graph classification, graph set representation, and image classification. Experimental results show that applying our ROTP layers can reduce the difficulty of the design and selection of global pooling -- our ROTP layers may either imitate some existing global pooling methods or lead to some new pooling layers fitting data better. The code is available at \url{https://github.com/SDS-Lab/ROT-Pooling}

    Structured ambiguity sets for distributionally robust optimization

    Full text link
    Distributionally robust optimization (DRO) incorporates robustness against uncertainty in the specification of probabilistic models. This paper focuses on mitigating the curse of dimensionality in data-driven DRO problems with optimal transport ambiguity sets. By exploiting independence across lower-dimensional components of the uncertainty, we construct structured ambiguity sets that exhibit a faster shrinkage as the number of collected samples increases. This narrows down the plausible models of the data-generating distribution and mitigates the conservativeness that the decisions of DRO problems over such ambiguity sets may face. We establish statistical guarantees for these structured ambiguity sets and provide dual reformulations of their associated DRO problems for a wide range of objective functions. The benefits of the approach are demonstrated in a numerical example
    • …
    corecore