13 research outputs found
MILO: Model-Agnostic Subset Selection Framework for Efficient Model Training and Tuning
Training deep networks and tuning hyperparameters on large datasets is
computationally intensive. One of the primary research directions for efficient
training is to reduce training costs by selecting well-generalizable subsets of
training data. Compared to simple adaptive random subset selection baselines,
existing intelligent subset selection approaches are not competitive due to the
time-consuming subset selection step, which involves computing model-dependent
gradients and feature embeddings and applies greedy maximization of submodular
objectives. Our key insight is that removing the reliance on downstream model
parameters enables subset selection as a pre-processing step and enables one to
train multiple models at no additional cost. In this work, we propose MILO, a
model-agnostic subset selection framework that decouples the subset selection
from model training while enabling superior model convergence and performance
by using an easy-to-hard curriculum. Our empirical results indicate that MILO
can train models faster and tune hyperparameters
faster than full-dataset training or tuning without
compromising performance
LakeBench: Benchmarks for Data Discovery over Data Lakes
Within enterprises, there is a growing need to intelligently navigate data
lakes, specifically focusing on data discovery. Of particular importance to
enterprises is the ability to find related tables in data repositories. These
tables can be unionable, joinable, or subsets of each other. There is a dearth
of benchmarks for these tasks in the public domain, with related work targeting
private datasets. In LakeBench, we develop multiple benchmarks for these tasks
by using the tables that are drawn from a diverse set of data sources such as
government data from CKAN, Socrata, and the European Central Bank. We compare
the performance of 4 publicly available tabular foundational models on these
tasks. None of the existing models had been trained on the data discovery tasks
that we developed for this benchmark; not surprisingly, their performance shows
significant room for improvement. The results suggest that the establishment of
such benchmarks may be useful to the community to build tabular models usable
for data discovery in data lakes