16 research outputs found
Anticipating Visual Representations from Unlabeled Video
Anticipating actions and objects before they start or appear is a difficult
problem in computer vision with several real-world applications. This task is
challenging partly because it requires leveraging extensive knowledge of the
world that is difficult to write down. We believe that a promising resource for
efficiently learning this knowledge is through readily available unlabeled
video. We present a framework that capitalizes on temporal structure in
unlabeled video to learn to anticipate human actions and objects. The key idea
behind our approach is that we can train deep networks to predict the visual
representation of images in the future. Visual representations are a promising
prediction target because they encode images at a higher semantic level than
pixels yet are automatic to compute. We then apply recognition algorithms on
our predicted representation to anticipate objects and actions. We
experimentally validate this idea on two datasets, anticipating actions one
second in the future and objects five seconds in the future.Comment: CVPR 201
AdaGraph: Unifying Predictive and Continuous Domain Adaptation through Graphs
The ability to categorize is a cornerstone of visual intelligence, and a key
functionality for artificial, autonomous visual machines. This problem will
never be solved without algorithms able to adapt and generalize across visual
domains. Within the context of domain adaptation and generalization, this paper
focuses on the predictive domain adaptation scenario, namely the case where no
target data are available and the system has to learn to generalize from
annotated source images plus unlabeled samples with associated metadata from
auxiliary domains. Our contributionis the first deep architecture that tackles
predictive domainadaptation, able to leverage over the information broughtby
the auxiliary domains through a graph. Moreover, we present a simple yet
effective strategy that allows us to take advantage of the incoming target data
at test time, in a continuous domain adaptation scenario. Experiments on three
benchmark databases support the value of our approach.Comment: CVPR 2019 (oral
Just in Time: Personal Temporal Insights for Altering Model Decisions
The interpretability of complex Machine Learning models is coming to be a
critical social concern, as they are increasingly used in human-related
decision-making processes such as resume filtering or loan applications.
Individuals receiving an undesired classification are likely to call for an
explanation -- preferably one that specifies what they should do in order to
alter that decision when they reapply in the future. Existing work focuses on a
single ML model and a single point in time, whereas in practice, both models
and data evolve over time: an explanation for an application rejection in 2018
may be irrelevant in 2019 since in the meantime both the model and the
applicant's data can change. To this end, we propose a novel framework that
provides users with insights and plans for changing their classification in
particular future time points. The solution is based on combining
state-of-the-art algorithms for (single) model explanations, ones for
predicting future models, and database-style querying of the obtained
explanations. We propose to demonstrate the usefulness of our solution in the
context of loan applications, and interactively engage the audience in
computing and viewing suggestions tailored for applicants based on their unique
characteristic
Multivariate Regression on the Grassmannian for Predicting Novel Domains
This work was supported by EPSRC (EP/L023385/1), and the European Unionâs Horizon 2020 research and innovation program under grant agreement No 640891