5,677 research outputs found
Visually-Aware Fashion Recommendation and Design with Generative Image Models
Building effective recommender systems for domains like fashion is
challenging due to the high level of subjectivity and the semantic complexity
of the features involved (i.e., fashion styles). Recent work has shown that
approaches to `visual' recommendation (e.g.~clothing, art, etc.) can be made
more accurate by incorporating visual signals directly into the recommendation
objective, using `off-the-shelf' feature representations derived from deep
networks. Here, we seek to extend this contribution by showing that
recommendation performance can be significantly improved by learning `fashion
aware' image representations directly, i.e., by training the image
representation (from the pixel level) and the recommender system jointly; this
contribution is related to recent work using Siamese CNNs, though we are able
to show improvements over state-of-the-art recommendation techniques such as
BPR and variants that make use of pre-trained visual features. Furthermore, we
show that our model can be used \emph{generatively}, i.e., given a user and a
product category, we can generate new images (i.e., clothing items) that are
most consistent with their personal taste. This represents a first step towards
building systems that go beyond recommending existing items from a product
corpus, but which can be used to suggest styles and aid the design of new
products.Comment: 10 pages, 6 figures. Accepted by ICDM'17 as a long pape
Complete the Look: Scene-based Complementary Product Recommendation
Modeling fashion compatibility is challenging due to its complexity and
subjectivity. Existing work focuses on predicting compatibility between product
images (e.g. an image containing a t-shirt and an image containing a pair of
jeans). However, these approaches ignore real-world 'scene' images (e.g.
selfies); such images are hard to deal with due to their complexity, clutter,
variations in lighting and pose (etc.) but on the other hand could potentially
provide key context (e.g. the user's body type, or the season) for making more
accurate recommendations. In this work, we propose a new task called 'Complete
the Look', which seeks to recommend visually compatible products based on scene
images. We design an approach to extract training data for this task, and
propose a novel way to learn the scene-product compatibility from fashion or
interior design images. Our approach measures compatibility both globally and
locally via CNNs and attention mechanisms. Extensive experiments show that our
method achieves significant performance gains over alternative systems. Human
evaluation and qualitative analysis are also conducted to further understand
model behavior. We hope this work could lead to useful applications which link
large corpora of real-world scenes with shoppable products.Comment: Accepted to CVPR'1
Explainable Fashion Recommendation: A Semantic Attribute Region Guided Approach
In fashion recommender systems, each product usually consists of multiple
semantic attributes (e.g., sleeves, collar, etc). When making cloth decisions,
people usually show preferences for different semantic attributes (e.g., the
clothes with v-neck collar). Nevertheless, most previous fashion recommendation
models comprehend the clothing images with a global content representation and
lack detailed understanding of users' semantic preferences, which usually leads
to inferior recommendation performance. To bridge this gap, we propose a novel
Semantic Attribute Explainable Recommender System (SAERS). Specifically, we
first introduce a fine-grained interpretable semantic space. We then develop a
Semantic Extraction Network (SEN) and Fine-grained Preferences Attention (FPA)
module to project users and items into this space, respectively. With SAERS, we
are capable of not only providing cloth recommendations for users, but also
explaining the reason why we recommend the cloth through intuitive visual
attribute semantic highlights in a personalized manner. Extensive experiments
conducted on real-world datasets clearly demonstrate the effectiveness of our
approach compared with the state-of-the-art methods.Comment: Accepted to IJCAI201
POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion
Increasing demand for fashion recommendation raises a lot of challenges for
online shopping platforms and fashion communities. In particular, there exist
two requirements for fashion outfit recommendation: the Compatibility of the
generated fashion outfits, and the Personalization in the recommendation
process. In this paper, we demonstrate these two requirements can be satisfied
via building a bridge between outfit generation and recommendation. Through
large data analysis, we observe that people have similar tastes in individual
items and outfits. Therefore, we propose a Personalized Outfit Generation (POG)
model, which connects user preferences regarding individual items and outfits
with Transformer architecture. Extensive offline and online experiments provide
strong quantitative evidence that our method outperforms alternative methods
regarding both compatibility and personalization metrics. Furthermore, we
deploy POG on a platform named Dida in Alibaba to generate personalized outfits
for the users of the online application iFashion.
This work represents a first step towards an industrial-scale fashion outfit
generation and recommendation solution, which goes beyond generating outfits
based on explicit queries, or merely recommending from existing outfit pools.
As part of this work, we release a large-scale dataset consisting of 1.01
million outfits with rich context information, and 0.28 billion user click
actions from 3.57 million users. To the best of our knowledge, this dataset is
the largest, publicly available, fashion related dataset, and the first to
provide user behaviors relating to both outfits and fashion items.Comment: Till appear in KDD 201
Compatible and Diverse Fashion Image Inpainting
Visual compatibility is critical for fashion analysis, yet is missing in
existing fashion image synthesis systems. In this paper, we propose to
explicitly model visual compatibility through fashion image inpainting. To this
end, we present Fashion Inpainting Networks (FiNet), a two-stage image-to-image
generation framework that is able to perform compatible and diverse inpainting.
Disentangling the generation of shape and appearance to ensure photorealistic
results, our framework consists of a shape generation network and an appearance
generation network. More importantly, for each generation network, we introduce
two encoders interacting with one another to learn latent code in a shared
compatibility space. The latent representations are jointly optimized with the
corresponding generation network to condition the synthesis process,
encouraging a diverse set of generated results that are visually compatible
with existing fashion garments. In addition, our framework is readily extended
to clothing reconstruction and fashion transfer, with impressive results.
Extensive experiments with comparisons with state-of-the-art approaches on
fashion synthesis task quantitatively and qualitatively demonstrate the
effectiveness of our method
Garment Design with Generative Adversarial Networks
The designers' tendency to adhere to a specific mental set and heavy
emotional investment in their initial ideas often hinder their ability to
innovate during the design thinking and ideation process. In the fashion
industry, in particular, the growing diversity of customers' needs, the intense
global competition, and the shrinking time-to-market (a.k.a., "fast fashion")
further exacerbate this challenge for designers. Recent advances in deep
generative models have created new possibilities to overcome the cognitive
obstacles of designers through automated generation and/or editing of design
concepts. This paper explores the capabilities of generative adversarial
networks (GAN) for automated attribute-level editing of design concepts.
Specifically, attribute GAN (AttGAN)---a generative model proven successful for
attribute editing of human faces---is utilized for automated editing of the
visual attributes of garments and tested on a large fashion dataset. The
experiments support the hypothesized potentials of GAN for attribute-level
editing of design concepts, and underscore several key limitations and research
questions to be addressed in future work.Comment: AdvML 2020, KDD worksho
A survey on Adversarial Recommender Systems: from Attack/Defense strategies to Generative Adversarial Networks
Latent-factor models (LFM) based on collaborative filtering (CF), such as
matrix factorization (MF) and deep CF methods, are widely used in modern
recommender systems (RS) due to their excellent performance and recommendation
accuracy. However, success has been accompanied with a major new arising
challenge: many applications of machine learning (ML) are adversarial in
nature. In recent years, it has been shown that these methods are vulnerable to
adversarial examples, i.e., subtle but non-random perturbations designed to
force recommendation models to produce erroneous outputs.
The goal of this survey is two-fold: (i) to present recent advances on
adversarial machine learning (AML) for the security of RS (i.e., attacking and
defense recommendation models), (ii) to show another successful application of
AML in generative adversarial networks (GANs) for generative applications,
thanks to their ability for learning (high-dimensional) data distributions. In
this survey, we provide an exhaustive literature review of 74 articles
published in major RS and ML journals and conferences. This review serves as a
reference for the RS community, working on the security of RS or on generative
models using GANs to improve their quality.Comment: 37 pages, submitted to journa
Adversarial Recommendation: Attack of the Learned Fake Users
Can machine learning models for recommendation be easily fooled? While the
question has been answered for hand-engineered fake user profiles, it has not
been explored for machine learned adversarial attacks. This paper attempts to
close this gap.
We propose a framework for generating fake user profiles which, when
incorporated in the training of a recommendation system, can achieve an
adversarial intent, while remaining indistinguishable from real user profiles.
We formulate this procedure as a repeated general-sum game between two players:
an oblivious recommendation system and an adversarial fake user generator
with two goals: (G1) the rating distribution of the fake users needs to be
close to the real users, and (G2) some objective encoding the attack
intent, such as targeting the top-K recommendation quality of for a subset
of users, needs to be optimized. We propose a learning framework to achieve
both goals, and offer extensive experiments considering multiple types of
attacks highlighting the vulnerability of recommendation systems
CuratorNet: Visually-aware Recommendation of Art Images
Although there are several visually-aware recommendation models in domains
like fashion or even movies, the art domain lacks thesame level of research
attention, despite the recent growth of the online artwork market. To reduce
this gap, in this article we introduceCuratorNet, a neural network architecture
for visually-aware recommendation of art images. CuratorNet is designed at the
core withthe goal of maximizing generalization: the network has a fixed set of
parameters that only need to be trained once, and thereafter themodel is able
to generalize to new users or items never seen before, without further
training. This is achieved by leveraging visualcontent: items are mapped to
item vectors through visual embeddings, and users are mapped to user vectors by
aggregating the visualcontent of items they have consumed. Besides the model
architecture, we also introduce novel triplet sampling strategies to build
atraining set for rank learning in the art domain, resulting in more effective
learning than naive random sampling. With an evaluationover a real-world
dataset of physical paintings, we show that CuratorNet achieves the best
performance among several baselines,including the state-of-the-art model VBPR.
CuratorNet is motivated and evaluated in the art domain, but its architecture
and trainingscheme could be adapted to recommend images in other area
Fashion++: Minimal Edits for Outfit Improvement
Given an outfit, what small changes would most improve its fashionability?
This question presents an intriguing new vision challenge. We introduce
Fashion++, an approach that proposes minimal adjustments to a full-body
clothing outfit that will have maximal impact on its fashionability. Our model
consists of a deep image generation neural network that learns to synthesize
clothing conditioned on learned per-garment encodings. The latent encodings are
explicitly factorized according to shape and texture, thereby allowing direct
edits for both fit/presentation and color/patterns/material, respectively. We
show how to bootstrap Web photos to automatically train a fashionability model,
and develop an activation maximization-style approach to transform the input
image into its more fashionable self. The edits suggested range from swapping
in a new garment to tweaking its color, how it is worn (e.g., rolling up
sleeves), or its fit (e.g., making pants baggier). Experiments demonstrate that
Fashion++ provides successful edits, both according to automated metrics and
human opinion. Project page is at
http://vision.cs.utexas.edu/projects/FashionPlus.Comment: accepted to ICCV 201
- …