3 research outputs found
In What Ways Are Deep Neural Networks Invariant and How Should We Measure This?
It is often said that a deep learning model is "invariant" to some specific
type of transformation. However, what is meant by this statement strongly
depends on the context in which it is made. In this paper we explore the nature
of invariance and equivariance of deep learning models with the goal of better
understanding the ways in which they actually capture these concepts on a
formal level. We introduce a family of invariance and equivariance metrics that
allows us to quantify these properties in a way that disentangles them from
other metrics such as loss or accuracy. We use our metrics to better understand
the two most popular methods used to build invariance into networks: data
augmentation and equivariant layers. We draw a range of conclusions about
invariance and equivariance in deep learning models, ranging from whether
initializing a model with pretrained weights has an effect on a trained model's
invariance, to the extent to which invariance learned via training can
generalize to out-of-distribution data.Comment: To appear at NeurIPS 202
ICML 2023 topological deep learning challenge. Design and results
This paper presents the computational challenge on topological deep learning that was hosted within the ICML 2023 Workshop on Topology and Geometry in Machine Learning. The competition asked participants to provide open-source implementations of topological neural networks from the literature by contributing to the python packages TopoNetX (data processing) and TopoModelX (deep learning). The challenge attracted twenty-eight qualifying submissions in its two-month duration. This paper describes the design of the challenge and summarizes its main finding