6,854 research outputs found
Styling with Attention to Details
Fashion as characterized by its nature, is driven by style. In this paper, we
propose a method that takes into account the style information to complete a
given set of selected fashion items with a complementary fashion item.
Complementary items are those items that can be worn along with the selected
items according to the style. Addressing this problem facilitates in
automatically generating stylish fashion ensembles leading to a richer shopping
experience for users.
Recently, there has been a surge of online social websites where fashion
enthusiasts post the outfit of the day and other users can like and comment on
them. These posts contain a gold-mine of information about style. In this
paper, we exploit these posts to train a deep neural network which captures
style in an automated manner. We pose the problem of predicting complementary
fashion items as a sequence to sequence problem where the input is the selected
set of fashion items and the output is a complementary fashion item based on
the style information learned by the model. We use the encoder decoder
architecture to solve this problem of completing the set of fashion items. We
evaluate the goodness of the proposed model through a variety of experiments.
We empirically observe that our proposed model outperforms competitive baseline
like apriori algorithm by ~28 in terms of accuracy for top-1 recommendation to
complete the fashion ensemble. We also perform retrieval based experiments to
understand the ability of the model to learn style and rank the complementary
fashion items and find that using attention in our encoder decoder model helps
in improving the mean reciprocal rank by ~24. Qualitatively we find the
complementary fashion items generated by our proposed model are richer than the
apriori algorithm
Aesthetic-based Clothing Recommendation
Recently, product images have gained increasing attention in clothing
recommendation since the visual appearance of clothing products has a
significant impact on consumers' decision. Most existing methods rely on
conventional features to represent an image, such as the visual features
extracted by convolutional neural networks (CNN features) and the
scale-invariant feature transform algorithm (SIFT features), color histograms,
and so on. Nevertheless, one important type of features, the \emph{aesthetic
features}, is seldom considered. It plays a vital role in clothing
recommendation since a users' decision depends largely on whether the clothing
is in line with her aesthetics, however the conventional image features cannot
portray this directly. To bridge this gap, we propose to introduce the
aesthetic information, which is highly relevant with user preference, into
clothing recommender systems. To achieve this, we first present the aesthetic
features extracted by a pre-trained neural network, which is a brain-inspired
deep structure trained for the aesthetic assessment task. Considering that the
aesthetic preference varies significantly from user to user and by time, we
then propose a new tensor factorization model to incorporate the aesthetic
features in a personalized manner. We conduct extensive experiments on
real-world datasets, which demonstrate that our approach can capture the
aesthetic preference of users and significantly outperform several
state-of-the-art recommendation methods.Comment: WWW 201
Fine-grained Apparel Classification and Retrieval without rich annotations
The ability to correctly classify and retrieve apparel images has a variety
of applications important to e-commerce, online advertising and internet
search. In this work, we propose a robust framework for fine-grained apparel
classification, in-shop and cross-domain retrieval which eliminates the
requirement of rich annotations like bounding boxes and human-joints or
clothing landmarks, and training of bounding box/ key-landmark detector for the
same. Factors such as subtle appearance differences, variations in human poses,
different shooting angles, apparel deformations, and self-occlusion add to the
challenges in classification and retrieval of apparel items. Cross-domain
retrieval is even harder due to the presence of large variation between online
shopping images, usually taken in ideal lighting, pose, positive angle and
clean background as compared with street photos captured by users in
complicated conditions with poor lighting and cluttered scenes. Our framework
uses compact bilinear CNN with tensor sketch algorithm to generate embeddings
that capture local pairwise feature interactions in a translationally invariant
manner. For apparel classification, we pass the feature embeddings through a
softmax classifier, while, the in-shop and cross-domain retrieval pipelines use
a triplet-loss based optimization approach, such that squared Euclidean
distance between embeddings measures the dissimilarity between the images.
Unlike previous works that relied on bounding box, key clothing landmarks or
human joint detectors to assist the final deep classifier, proposed framework
can be trained directly on the provided category labels or generated triplets
for triplet loss optimization. Lastly, Experimental results on the DeepFashion
fine-grained categorization, and in-shop and consumer-to-shop retrieval
datasets provide a comparative analysis with previous work performed in the
domain.Comment: 14 pages, 6 figures, 3 tables, Submitted to Springer Journal of
Applied Intelligenc
Deep Style Match for Complementary Recommendation
Humans develop a common sense of style compatibility between items based on
their attributes. We seek to automatically answer questions like "Does this
shirt go well with that pair of jeans?" In order to answer these kinds of
questions, we attempt to model human sense of style compatibility in this
paper. The basic assumption of our approach is that most of the important
attributes for a product in an online store are included in its title
description. Therefore it is feasible to learn style compatibility from these
descriptions. We design a Siamese Convolutional Neural Network architecture and
feed it with title pairs of items, which are either compatible or incompatible.
Those pairs will be mapped from the original space of symbolic words into some
embedded style space. Our approach takes only words as the input with few
preprocessing and there is no laborious and expensive feature engineering.Comment: Workshops at the Thirty-First AAAI Conference on Artificial
Intelligenc
Visually-aware Recommendation with Aesthetic Features
Visual information plays a critical role in human decision-making process.
While recent developments on visually-aware recommender systems have taken the
product image into account, none of them has considered the aesthetic aspect.
We argue that the aesthetic factor is very important in modeling and predicting
users' preferences, especially for some fashion-related domains like clothing
and jewelry. This work addresses the need of modeling aesthetic information in
visually-aware recommender systems. Technically speaking, we make three key
contributions in leveraging deep aesthetic features: (1) To describe the
aesthetics of products, we introduce the aesthetic features extracted from
product images by a deep aesthetic network. We incorporate these features into
recommender system to model users' preferences in the aesthetic aspect. (2)
Since in clothing recommendation, time is very important for users to make
decision, we design a new tensor decomposition model for implicit feedback
data. The aesthetic features are then injected to the basic tensor model to
capture the temporal dynamics of aesthetic preferences (e.g., seasonal
patterns). (3) We also use the aesthetic features to optimize the learning
strategy on implicit feedback data. We enrich the pairwise training samples by
considering the similarity among items in the visual space and graph space; the
key idea is that a user may likely have similar perception on similar items. We
perform extensive experiments on several real-world datasets and demonstrate
the usefulness of aesthetic features and the effectiveness of our proposed
methods.Comment: Accepted by VLDBJ. arXiv admin note: substantial text overlap with
arXiv:1809.0582
Using Artificial Intelligence to Analyze Fashion Trends
Analyzing fashion trends is essential in the fashion industry. Current
fashion forecasting firms, such as WGSN, utilize the visual information from
around the world to analyze and predict fashion trends. However, analyzing
fashion trends is time-consuming and extremely labor intensive, requiring
individual employees' manual editing and classification. To improve the
efficiency of data analysis of such image-based information and lower the cost
of analyzing fashion images, this study proposes a data-driven quantitative
abstracting approach using an artificial intelligence (A.I.) algorithm.
Specifically, an A.I. model was trained on fashion images from a large-scale
dataset under different scenarios, for example in online stores and street
snapshots. This model was used to detect garments and classify clothing
attributes such as textures, garment style, and details for runway photos and
videos. It was found that the A.I. model can generate rich attribute
descriptions of detected regions and accurately bind the garments in the
images. Adoption of A.I. algorithm demonstrated promising results and the
potential to classify garment types and details automatically, which can make
the process of trend forecasting more cost-effective and faster
DeepStyle: Multimodal Search Engine for Fashion and Interior Design
In this paper, we propose a multimodal search engine that combines visual and
textual cues to retrieve items from a multimedia database aesthetically similar
to the query. The goal of our engine is to enable intuitive retrieval of
fashion merchandise such as clothes or furniture. Existing search engines treat
textual input only as an additional source of information about the query image
and do not correspond to the real-life scenario where the user looks for 'the
same shirt but of denim'. Our novel method, dubbed DeepStyle, mitigates those
shortcomings by using a joint neural network architecture to model contextual
dependencies between features of different modalities. We prove the robustness
of this approach on two different challenging datasets of fashion items and
furniture where our DeepStyle engine outperforms baseline methods by 18-21% on
the tested datasets. Our search engine is commercially deployed and available
through a Web-based application.Comment: Copyright held by IEEE. Personal use of this material is permitted.
Permission from IEEE must be obtained for all other uses, in any current or
future media, including reprinting/republishing this material for advertising
or promotional purposes, creating new collective works, for resale or
redistribution to servers or lists, or reuse of any copyrighted component of
this work in other work
Spectrum-enhanced Pairwise Learning to Rank
To enhance the performance of the recommender system, side information is
extensively explored with various features (e.g., visual features and textual
features). However, there are some demerits of side information: (1) the extra
data is not always available in all recommendation tasks; (2) it is only for
items, there is seldom high-level feature describing users. To address these
gaps, we introduce the spectral features extracted from two hypergraph
structures of the purchase records. Spectral features describe the
\textit{similarity} of users/items in the graph space, which is critical for
recommendation. We leverage spectral features to model the users' preference
and items' properties by incorporating them into a Matrix Factorization (MF)
model. In addition to modeling, we also use spectral features to optimize.
Bayesian Personalized Ranking (BPR) is extensively leveraged to optimize models
in implicit feedback data. However, in BPR, all missing values are regarded as
negative samples equally while many of them are indeed unseen positive ones. We
enrich the positive samples by calculating the similarity among users/items by
the spectral features. The key ideas are: (1) similar users shall have similar
preference on the same item; (2) a user shall have similar perception on
similar items. Extensive experiments on two real-world datasets demonstrate the
usefulness of the spectral features and the effectiveness of our
spectrum-enhanced pairwise optimization. Our models outperform several
state-of-the-art models significantly.Comment: 11 pages; submitted to World Wide Web Conference (WWW 2019
HCRS: A hybrid clothes recommender system based on user ratings and product features
Nowadays, online clothes-selling business has become popular and extremely
attractive because of its convenience and cheap-and-fine price. Good examples
of these successful Web sites include Yintai.com, Vancl.com and
Shop.vipshop.com which provide thousands of clothes for online shoppers. The
challenge for online shoppers lies on how to find a good product from lots of
options. In this article, we propose a collaborative clothes recommender for
easy shopping. One of the unique features of this system is the ability to
recommend clothes in terms of both user ratings and clothing attributes.
Experiments in our simulation environment show that the proposed recommender
can better satisfy the needs of users.Comment: ICMECG '13 Proceedings of the 2013 International Conference on
Management of e-Commerce and e-Government Pages 270-27
Complete the Look: Scene-based Complementary Product Recommendation
Modeling fashion compatibility is challenging due to its complexity and
subjectivity. Existing work focuses on predicting compatibility between product
images (e.g. an image containing a t-shirt and an image containing a pair of
jeans). However, these approaches ignore real-world 'scene' images (e.g.
selfies); such images are hard to deal with due to their complexity, clutter,
variations in lighting and pose (etc.) but on the other hand could potentially
provide key context (e.g. the user's body type, or the season) for making more
accurate recommendations. In this work, we propose a new task called 'Complete
the Look', which seeks to recommend visually compatible products based on scene
images. We design an approach to extract training data for this task, and
propose a novel way to learn the scene-product compatibility from fashion or
interior design images. Our approach measures compatibility both globally and
locally via CNNs and attention mechanisms. Extensive experiments show that our
method achieves significant performance gains over alternative systems. Human
evaluation and qualitative analysis are also conducted to further understand
model behavior. We hope this work could lead to useful applications which link
large corpora of real-world scenes with shoppable products.Comment: Accepted to CVPR'1
- …