55,813 research outputs found
A Graph Theoretic Approach for Object Shape Representation in Compositional Hierarchies Using a Hybrid Generative-Descriptive Model
A graph theoretic approach is proposed for object shape representation in a
hierarchical compositional architecture called Compositional Hierarchy of Parts
(CHOP). In the proposed approach, vocabulary learning is performed using a
hybrid generative-descriptive model. First, statistical relationships between
parts are learned using a Minimum Conditional Entropy Clustering algorithm.
Then, selection of descriptive parts is defined as a frequent subgraph
discovery problem, and solved using a Minimum Description Length (MDL)
principle. Finally, part compositions are constructed by compressing the
internal data representation with discovered substructures. Shape
representation and computational complexity properties of the proposed approach
and algorithms are examined using six benchmark two-dimensional shape image
datasets. Experiments show that CHOP can employ part shareability and indexing
mechanisms for fast inference of part compositions using learned shape
vocabularies. Additionally, CHOP provides better shape retrieval performance
than the state-of-the-art shape retrieval methods.Comment: Paper : 17 pages. 13th European Conference on Computer Vision (ECCV
2014), Zurich, Switzerland, September 6-12, 2014, Proceedings, Part III, pp
566-581. Supplementary material can be downloaded from
http://link.springer.com/content/esm/chp:10.1007/978-3-319-10578-9_37/file/MediaObjects/978-3-319-10578-9_37_MOESM1_ESM.pd
GIPCOL: Graph-Injected Soft Prompting for Compositional Zero-Shot Learning
Pre-trained vision-language models (VLMs) have achieved promising success in
many fields, especially with prompt learning paradigm. In this work, we propose
GIP-COL (Graph-Injected Soft Prompting for COmpositional Learning) to better
explore the compositional zero-shot learning (CZSL) ability of VLMs within the
prompt-based learning framework. The soft prompt in GIPCOL is structured and
consists of the prefix learnable vectors, attribute label and object label. In
addition, the attribute and object labels in the soft prompt are designated as
nodes in a compositional graph. The compositional graph is constructed based on
the compositional structure of the objects and attributes extracted from the
training data and consequently feeds the updated concept representation into
the soft prompt to capture this compositional structure for a better prompting
for CZSL. With the new prompting strategy, GIPCOL achieves state-of-the-art AUC
results on all three CZSL benchmarks, including MIT-States, UT-Zappos, and
C-GQA datasets in both closed and open settings compared to previous non-CLIP
as well as CLIP-based methods. We analyze when and why GIPCOL operates well
given the CLIP backbone and its training data limitations, and our findings
shed light on designing more effective prompts for CZSLComment: WACV2
Semantic Representation and Inference for NLP
Semantic representation and inference is essential for Natural Language
Processing (NLP). The state of the art for semantic representation and
inference is deep learning, and particularly Recurrent Neural Networks (RNNs),
Convolutional Neural Networks (CNNs), and transformer Self-Attention models.
This thesis investigates the use of deep learning for novel semantic
representation and inference, and makes contributions in the following three
areas: creating training data, improving semantic representations and extending
inference learning. In terms of creating training data, we contribute the
largest publicly available dataset of real-life factual claims for the purpose
of automatic claim verification (MultiFC), and we present a novel inference
model composed of multi-scale CNNs with different kernel sizes that learn from
external sources to infer fact checking labels. In terms of improving semantic
representations, we contribute a novel model that captures non-compositional
semantic indicators. By definition, the meaning of a non-compositional phrase
cannot be inferred from the individual meanings of its composing words (e.g.,
hot dog). Motivated by this, we operationalize the compositionality of a phrase
contextually by enriching the phrase representation with external word
embeddings and knowledge graphs. Finally, in terms of inference learning, we
propose a series of novel deep learning architectures that improve inference by
using syntactic dependencies, by ensembling role guided attention heads,
incorporating gating layers, and concatenating multiple heads in novel and
effective ways. This thesis consists of seven publications (five published and
two under review).Comment: PhD thesis, the University of Copenhage
A Theory of Emergent In-Context Learning as Implicit Structure Induction
Scaling large language models (LLMs) leads to an emergent capacity to learn
in-context from example demonstrations. Despite progress, theoretical
understanding of this phenomenon remains limited. We argue that in-context
learning relies on recombination of compositional operations found in natural
language data. We derive an information-theoretic bound showing how in-context
learning abilities arise from generic next-token prediction when the
pretraining distribution has sufficient amounts of compositional structure,
under linguistically motivated assumptions. A second bound provides a
theoretical justification for the empirical success of prompting LLMs to output
intermediate steps towards an answer. To validate theoretical predictions, we
introduce a controlled setup for inducing in-context learning; unlike previous
approaches, it accounts for the compositional nature of language. Trained
transformers can perform in-context learning for a range of tasks, in a manner
consistent with the theoretical results. Mirroring real-world LLMs in a
miniature setup, in-context learning emerges when scaling parameters and data,
and models perform better when prompted to output intermediate steps. Probing
shows that in-context learning is supported by a representation of the input's
compositional structure. Taken together, these results provide a step towards
theoretical understanding of emergent behavior in large language models
3D ShapeNets: A Deep Representation for Volumetric Shapes
3D shape is a crucial but heavily underutilized cue in today's computer
vision systems, mostly due to the lack of a good generic shape representation.
With the recent availability of inexpensive 2.5D depth sensors (e.g. Microsoft
Kinect), it is becoming increasingly important to have a powerful 3D shape
representation in the loop. Apart from category recognition, recovering full 3D
shapes from view-based 2.5D depth maps is also a critical part of visual
understanding. To this end, we propose to represent a geometric 3D shape as a
probability distribution of binary variables on a 3D voxel grid, using a
Convolutional Deep Belief Network. Our model, 3D ShapeNets, learns the
distribution of complex 3D shapes across different object categories and
arbitrary poses from raw CAD data, and discovers hierarchical compositional
part representations automatically. It naturally supports joint object
recognition and shape completion from 2.5D depth maps, and it enables active
object recognition through view planning. To train our 3D deep learning model,
we construct ModelNet -- a large-scale 3D CAD model dataset. Extensive
experiments show that our 3D deep representation enables significant
performance improvement over the-state-of-the-arts in a variety of tasks.Comment: to be appeared in CVPR 201
RNNs Implicitly Implement Tensor Product Representations
Recurrent neural networks (RNNs) can learn continuous vector representations
of symbolic structures such as sequences and sentences; these representations
often exhibit linear regularities (analogies). Such regularities motivate our
hypothesis that RNNs that show such regularities implicitly compile symbolic
structures into tensor product representations (TPRs; Smolensky, 1990), which
additively combine tensor products of vectors representing roles (e.g.,
sequence positions) and vectors representing fillers (e.g., particular words).
To test this hypothesis, we introduce Tensor Product Decomposition Networks
(TPDNs), which use TPRs to approximate existing vector representations. We
demonstrate using synthetic data that TPDNs can successfully approximate linear
and tree-based RNN autoencoder representations, suggesting that these
representations exhibit interpretable compositional structure; we explore the
settings that lead RNNs to induce such structure-sensitive representations. By
contrast, further TPDN experiments show that the representations of four models
trained to encode naturally-occurring sentences can be largely approximated
with a bag of words, with only marginal improvements from more sophisticated
structures. We conclude that TPDNs provide a powerful method for interpreting
vector representations, and that standard RNNs can induce compositional
sequence representations that are remarkably well approximated by TPRs; at the
same time, existing training tasks for sentence representation learning may not
be sufficient for inducing robust structural representations.Comment: Accepted to ICLR 201
- …