Representation Reliability and Its Impact on Downstream Tasks

Self-supervised pre-trained models extract general-purpose representations from data, and quantifying how reliable they are is crucial because many downstream models use these representations as input for their own tasks. To this end, we first introduce a formal definition of representation reliability: the representation for a given test input is considered to be reliable if the downstream models built on top of that representation can consistently generate accurate predictions for that test point. It is desired to estimate the representation reliability without knowing the downstream tasks a priori. We provide a negative result showing that existing frameworks for uncertainty quantification in supervised learning are not suitable for this purpose. As an alternative, we propose an ensemble-based method for quantifying representation reliability, based on the concept of neighborhood consistency in the representation spaces across various pre-trained models. More specifically, the key insight is to use shared neighboring points as anchors to align different representation spaces. We demonstrate through comprehensive numerical experiments that our method is capable of predicting representation reliability with high accuracy

Some Results on Skorokhod Embedding and Robust Hedging with Local Time

In this paper, we provide some results on Skorokhod embedding with local time and its applications to the robust hedging problem in finance. First we investigate the robust hedging of options depending on the local time by using the recently introduced stochastic control approach, in order to identify the optimal hedging strategies, as well as the market models that realize the extremal no-arbitrage prices. As a by-product, the optimality of Vallois' Skorokhod embeddings is recovered. In addition, under appropriate conditions, we derive a new solution to the two-marginal Skorokhod embedding as a generalization of the Vallois solution. It turns out from our analysis that one needs to relax the monotonicity assumption on the embedding functions in order to embed a larger class of marginal distributions. Finally, in a full-marginal setting where the stopping times given by Vallois are well-ordered, we construct a remarkable Markov martingale which provides a new example of fake Brownian motion

Probabilistic Concept Bottleneck Models

Interpretable models are designed to make decisions in a human-interpretable manner. Representatively, Concept Bottleneck Models (CBM) follow a two-step process of concept prediction and class prediction based on the predicted concepts. CBM provides explanations with high-level concepts derived from concept predictions; thus, reliable concept predictions are important for trustworthiness. In this study, we address the ambiguity issue that can harm reliability. While the existence of a concept can often be ambiguous in the data, CBM predicts concepts deterministically without considering this ambiguity. To provide a reliable interpretation against this ambiguity, we propose Probabilistic Concept Bottleneck Models (ProbCBM). By leveraging probabilistic concept embeddings, ProbCBM models uncertainty in concept prediction and provides explanations based on the concept and its corresponding uncertainty. This uncertainty enhances the reliability of the explanations. Furthermore, as class uncertainty is derived from concept uncertainty in ProbCBM, we can explain class uncertainty by means of concept uncertainty. Code is publicly available at https://github.com/ejkim47/prob-cbm.Comment: International Conference on Machine Learning (ICML) 202

Introspective Deep Metric Learning for Image Retrieval

This paper proposes an introspective deep metric learning (IDML) framework for uncertainty-aware comparisons of images. Conventional deep metric learning methods produce confident semantic distances between images regardless of the uncertainty level. However, we argue that a good similarity model should consider the semantic discrepancies with caution to better deal with ambiguous images for more robust training. To achieve this, we propose to represent an image using not only a semantic embedding but also an accompanying uncertainty embedding, which describes the semantic characteristics and ambiguity of an image, respectively. We further propose an introspective similarity metric to make similarity judgments between images considering both their semantic differences and ambiguities. The proposed IDML framework improves the performance of deep metric learning through uncertainty modeling and attains state-of-the-art results on the widely used CUB-200-2011, Cars196, and Stanford Online Products datasets for image retrieval and clustering. We further provide an in-depth analysis of our framework to demonstrate the effectiveness and reliability of IDML. Code is available at: https://github.com/wzzheng/IDML.Comment: The extended version of this paper is accepted to T-PAMI. Source code available at https://github.com/wzzheng/IDM