1,112 research outputs found
Learning Aligned Cross-Modal Representations from Weakly Aligned Data
People can recognize scenes across many different modalities beyond natural
images. In this paper, we investigate how to learn cross-modal scene
representations that transfer across modalities. To study this problem, we
introduce a new cross-modal scene dataset. While convolutional neural networks
can categorize cross-modal scenes well, they also learn an intermediate
representation not aligned across modalities, which is undesirable for
cross-modal transfer applications. We present methods to regularize cross-modal
convolutional neural networks so that they have a shared representation that is
agnostic of the modality. Our experiments suggest that our scene representation
can help transfer representations across modalities for retrieval. Moreover,
our visualizations suggest that units emerge in the shared representation that
tend to activate on consistent concepts independently of the modality.Comment: Conference paper at CVPR 201
Recent Advances in Transfer Learning for Cross-Dataset Visual Recognition: A Problem-Oriented Perspective
This paper takes a problem-oriented perspective and presents a comprehensive
review of transfer learning methods, both shallow and deep, for cross-dataset
visual recognition. Specifically, it categorises the cross-dataset recognition
into seventeen problems based on a set of carefully chosen data and label
attributes. Such a problem-oriented taxonomy has allowed us to examine how
different transfer learning approaches tackle each problem and how well each
problem has been researched to date. The comprehensive problem-oriented review
of the advances in transfer learning with respect to the problem has not only
revealed the challenges in transfer learning for visual recognition, but also
the problems (e.g. eight of the seventeen problems) that have been scarcely
studied. This survey not only presents an up-to-date technical review for
researchers, but also a systematic approach and a reference for a machine
learning practitioner to categorise a real problem and to look up for a
possible solution accordingly
Understand-Before-Talk (UBT): A Semantic Communication Approach to 6G Networks
In Shannon theory, semantic aspects of communication were identified but
considered irrelevant to the technical communication problems. Semantic
communication (SC) techniques have recently attracted renewed research
interests in (6G) wireless because they have the capability to support an
efficient interpretation of the significance and meaning intended by a sender
(or accomplishment of the goal) when dealing with multi-modal data such as
videos, images, audio, text messages, and so on, which would be the case for
various applications such as intelligent transportation systems where each
autonomous vehicle needs to deal with real-time videos and data from a number
of sensors including radars. A notable difficulty of existing SC frameworks
lies in handling the discrete constraints imposed on the pursued semantic
coding and its interaction with the independent knowledge base, which makes
reliable semantic extraction extremely challenging. Therefore, we develop a new
lightweight hashing-based semantic extraction approach to the SC framework,
where our learning objective is to generate one-time signatures (hash codes)
using supervised learning for low latency, secure and efficient management of
the SC dynamics. We first evaluate the proposed semantic extraction framework
over large image data sets, extend it with domain adaptive hashing and then
demonstrate the effectiveness of "semantics signature" in bulk transmission and
multi-modal data
Binary Representation Learning for Large Scale Visual Data
The exponentially growing modern media created large amount of multimodal or multidomain visual data, which usually reside in high dimensional space. And it is crucial to provide not only effective but also efficient understanding of the data.In this dissertation, we focus on learning binary representation of visual dataset, whose primary use has been hash code for retrieval purpose. Simultaneously it serves as multifunctional feature that can also be used for various computer vision tasks. Essentially, this is achieved by discriminative learning that preserves the supervision information in the binary representation.By using deep networks such as convolutional neural networks (CNNs) as backbones, and effective binary embedding algorithm that is seamlessly integrated into the learning process, we achieve state-of-the art performance on several settings. First, we study the supervised binary representation learning problem by using label information directly instead of pairwise similarity or triplet loss. By considering images and associated textual information, we study the cross-modal representation learning. CNNs are used in both image and text embedding, and we are able to perform retrieval and prediction across these modalities. Furthermore, by utilizing unlabeled images from a different domain, we propose to use adversarial learning to connect these domains. Finally, we also consider progressive learning for more efficient learning and instance-level representation learning to provide finer granularity understanding. This dissertation demonstrates that binary representation is versatile and powerful under various circumstances with different tasks
- …