7 research outputs found
Towards Anytime Classification in Early-Exit Architectures by Enforcing Conditional Monotonicity
Modern predictive models are often deployed to environments in which
computational budgets are dynamic. Anytime algorithms are well-suited to such
environments as, at any point during computation, they can output a prediction
whose quality is a function of computation time. Early-exit neural networks
have garnered attention in the context of anytime computation due to their
capability to provide intermediate predictions at various stages throughout the
network. However, we demonstrate that current early-exit networks are not
directly applicable to anytime settings, as the quality of predictions for
individual data points is not guaranteed to improve with longer computation. To
address this shortcoming, we propose an elegant post-hoc modification, based on
the Product-of-Experts, that encourages an early-exit network to become
gradually confident. This gives our deep models the property of conditional
monotonicity in the prediction quality -- an essential stepping stone towards
truly anytime predictive modeling using early-exit architectures. Our empirical
results on standard image-classification tasks demonstrate that such behaviors
can be achieved while preserving competitive accuracy on average.Comment: NeurIPS 202
Anytime-Valid Confidence Sequences for Consistent Uncertainty Estimation in Early-Exit Neural Networks
Early-exit neural networks (EENNs) facilitate adaptive inference by producing
predictions at multiple stages of the forward pass. In safety-critical
applications, these predictions are only meaningful when complemented with
reliable uncertainty estimates. Yet, due to their sequential structure, an
EENN's uncertainty estimates should also be consistent: labels that are deemed
improbable at one exit should not reappear within the confidence interval / set
of later exits. We show that standard uncertainty quantification techniques,
like Bayesian methods or conformal prediction, can lead to inconsistency across
exits. We address this problem by applying anytime-valid confidence sequences
(AVCSs) to the exits of EENNs. By design, AVCSs maintain consistency across
exits. We examine the theoretical and practical challenges of applying AVCSs to
EENNs and empirically validate our approach on both regression and
classification tasks
General definition of differential privacy
V delu predstavimo diferencirano zasebnost. Gre za matematično definicijo zasebnosti pri javni objavi ter rudarjenju podatkov. Predstavljena je splošna definicija v kontekstu metričnih prostorov in verjetnostne mere, ki omogoča enotno obravnavo različnih vrst podatkov. Pokažemo nekaj osnovnih izrekov, ki omilijo zahteve definicije. Obravnavan je Laplaceov mehanizem za numerične podatke. Podana je izpeljava spodnjih mej za največjo napako zasebnih odzivnih mehanizmov. V nadaljevanju se osredotočimo na funkcijske podatke. S pomočjo teorije Gaussovih procesov in Hilbertovih prostorov z reprodukcijskim jedrom pokažemo uporabo diferencirane zasebnosti na primeru jedrne cenilke gostote. Osnovne mehanizme implementiramo in predstavimo rezultate.We introduce the concept of differential privacy, mathematical definition for privacy preserving data publishing and data mining. General definition in context of metric spaces and probability measure is given. Further, we present some theorems which help to alleviate the requirements of described definition. Laplace mechanism for numerical data and lower bounds on errors of response mechanisms are presented. We later turn focus to functional data. Using Gaussian processes and Reproducing Kernel Hilbert Spaces we present how differential privacy is used for privatization of density kernel estimator. Most of the described mechanisms are also implemented and results are presented at the en
On the impact of publicly available news and information transfer to financial markets
We quantify the propagation and absorption of large-scale publicly available news articles from the World Wide Web to financial markets. To extract publicly available information, we use the news archives from the Common Crawl, a non-profit organization that crawls a large part of the web. We develop a processing pipeline to identify news articles associated with the constituent companies in the S&P 500 index, an equity market index that measures the stock performance of US companies. Using machine learning techniques, we extract sentiment scores from the Common Crawl News data and employ tools from information theory to quantify the information transfer from public news articles to the US stock market. Furthermore, we analyse and quantify the economic significance of the news-based information with a simple sentiment-based portfolio trading strategy. Our findings provide support for that information in publicly available news on the World Wide Web has a statistically and economically significant impact on events in financial markets.ISSN:2054-570