344 research outputs found
Multitask Efficiencies in the Decision Tree Model
In Direct Sum problems [KRW], one tries to show that for a given
computational model, the complexity of computing a collection of finite
functions on independent inputs is approximately the sum of their individual
complexities. In this paper, by contrast, we study the diversity of ways in
which the joint computational complexity can behave when all the functions are
evaluated on a common input. We focus on the deterministic decision tree model,
with depth as the complexity measure; in this model we prove a result to the
effect that the 'obvious' constraints on joint computational complexity are
essentially the only ones.
The proof uses an intriguing new type of cryptographic data structure called
a `mystery bin' which we construct using a small polynomial separation between
deterministic and unambiguous query complexity shown by Savicky. We also pose a
variant of the Direct Sum Conjecture of [KRW] which, if proved for a single
family of functions, could yield an analogous result for models such as the
communication model.Comment: Improved exposition based on conference versio
Multitask Efficiencies in the Decision Tree Model
In Direct Sum problems [KRW], one tries to show that for a given
computational model, the complexity of computing a collection of finite
functions on independent inputs is approximately the sum of their individual
complexities. In this paper, by contrast, we study the diversity of ways in
which the joint computational complexity can behave when all the functions are
evaluated on a common input. We focus on the deterministic decision tree model,
with depth as the complexity measure; in this model we prove a result to the
effect that the 'obvious' constraints on joint computational complexity are
essentially the only ones.
The proof uses an intriguing new type of cryptographic data structure called
a `mystery bin' which we construct using a small polynomial separation between
deterministic and unambiguous query complexity shown by Savicky. We also pose a
variant of the Direct Sum Conjecture of [KRW] which, if proved for a single
family of functions, could yield an analogous result for models such as the
communication model.Comment: Improved exposition based on conference versio
Omnidirectional Transfer for Quasilinear Lifelong Learning
In biological learning, data are used to improve performance not only on the
current task, but also on previously encountered and as yet unencountered
tasks. In contrast, classical machine learning starts from a blank slate, or
tabula rasa, using data only for the single task at hand. While typical
transfer learning algorithms can improve performance on future tasks, their
performance on prior tasks degrades upon learning new tasks (called
catastrophic forgetting). Many recent approaches for continual or lifelong
learning have attempted to maintain performance given new tasks. But striving
to avoid forgetting sets the goal unnecessarily low: the goal of lifelong
learning, whether biological or artificial, should be to improve performance on
all tasks (including past and future) with any new data. We propose
omnidirectional transfer learning algorithms, which includes two special cases
of interest: decision forests and deep networks. Our key insight is the
development of the omni-voter layer, which ensembles representations learned
independently on all tasks to jointly decide how to proceed on any given new
data point, thereby improving performance on both past and future tasks. Our
algorithms demonstrate omnidirectional transfer in a variety of simulated and
real data scenarios, including tabular data, image data, spoken data, and
adversarial tasks. Moreover, they do so with quasilinear space and time
complexity
Machine Learning Toxicity Prediction: Latest Advances by Toxicity End Point
Machine learning (ML) models to predict the toxicity of small molecules have garnered great attention and have become widely used in recent years. Computational toxicity prediction is particularly advantageous in the early stages of drug discovery in order to filter out molecules with high probability of failing in clinical trials. This has been helped by the increase in the number of large toxicology databases available. However, being an area of recent application, a greater understanding of the scope and applicability of ML methods is still necessary. There are various kinds of toxic end points that have been predicted in silico. Acute oral toxicity, hepatotoxicity, cardiotoxicity, mutagenicity, and the 12 Tox21 data end points are among the most commonly investigated. Machine learning methods exhibit different performances on different data sets due to dissimilar complexity, class distributions, or chemical space covered, which makes it hard to compare the performance of algorithms over different toxic end points. The general pipeline to predict toxicity using ML has already been analyzed in various reviews. In this contribution, we focus on the recent progress in the area and the outstanding challenges, making a detailed description of the state-of-the-art models implemented for each toxic end point. The type of molecular representation, the algorithm, and the evaluation metric used in each research work are explained and analyzed. A detailed description of end points that are usually predicted, their clinical relevance, the available databases, and the challenges they bring to the field are also highlighted.Fil: Cavasotto, Claudio Norberto. Universidad Austral. Facultad de Ciencias Biomédicas. Instituto de Investigaciones en Medicina Traslacional. Consejo Nacional de Investigaciones CientÃficas y Técnicas. Oficina de Coordinación Administrativa Parque Centenario. Instituto de Investigaciones en Medicina Traslacional; ArgentinaFil: Scardino, Valeria. Universidad Austral; Argentin
Efficient Multiuser AI Downloading via Reusable Knowledge Broadcasting
For the 6G mobile networks, in-situ model downloading has emerged as an
important use case to enable real-time adaptive artificial intelligence on edge
devices. However, the simultaneous downloading of diverse and high-dimensional
models to multiple devices over wireless links presents a significant
communication bottleneck. To overcome the bottleneck, we propose the framework
of model broadcasting and assembling (MBA), which represents the first attempt
on leveraging reusable knowledge, referring to shared parameters among tasks,
to enable parameter broadcasting to reduce communication overhead. The MBA
framework comprises two key components. The first, the MBA protocol, defines
the system operations including parameter selection from a model library, power
control for broadcasting, and model assembling at devices. The second component
is the joint design of parameter-selection-and-power-control (PS-PC), which
provides guarantees on devices' model performance and minimizes the downloading
latency. The corresponding optimization problem is simplified by decomposition
into the sequential PS and PC sub-problems without compromising its optimality.
The PS sub-problem is solved efficiently by designing two efficient algorithms.
On one hand, the low-complexity algorithm of greedy parameter selection
features the construction of candidate model sets and a selection metric, both
of which are designed under the criterion of maximum reusable knowledge among
tasks. On the other hand, the optimal tree-search algorithm gains its
efficiency via the proposed construction of a compact binary tree pruned using
model architecture constraints and an intelligent branch-and-bound search.
Given optimal PS, the optimal PC policy is derived in closed form. Extensive
experiments demonstrate the substantial reduction in downloading latency
achieved by the proposed MBA compared to traditional model downloading.Comment: Submitted to IEEE for possible publicatio
- …