23 research outputs found
Tactile Mesh Saliency:a brief synopsis
This work has previously been published [LDS 16] and this extended abstract provides a synopsis for further discussion at the UK CGVC 2016 conference. We introduce the concept of tactile mesh saliency, where tactile salient points on a virtual mesh are those that a human is more likely to grasp, press, or touch if the mesh were a real-world object. We solve the problem of taking as input a 3D mesh and computing the tactile saliency of every mesh vertex. The key to solving this problem is in a new formulation that combines deep learning and learning-to-rank methods to compute a tactile saliency measure. Finally, we discuss possibilities for future work
A Similarity Measure for Material Appearance
We present a model to measure the similarity in appearance between different
materials, which correlates with human similarity judgments. We first create a
database of 9,000 rendered images depicting objects with varying materials,
shape and illumination. We then gather data on perceived similarity from
crowdsourced experiments; our analysis of over 114,840 answers suggests that
indeed a shared perception of appearance similarity exists. We feed this data
to a deep learning architecture with a novel loss function, which learns a
feature space for materials that correlates with such perceived appearance
similarity. Our evaluation shows that our model outperforms existing metrics.
Last, we demonstrate several applications enabled by our metric, including
appearance-based search for material suggestions, database visualization,
clustering and summarization, and gamut mapping.Comment: 12 pages, 17 figure
An intuitive control space for material appearance
Many different techniques for measuring material appearance have been
proposed in the last few years. These have produced large public datasets,
which have been used for accurate, data-driven appearance modeling. However,
although these datasets have allowed us to reach an unprecedented level of
realism in visual appearance, editing the captured data remains a challenge. In
this paper, we present an intuitive control space for predictable editing of
captured BRDF data, which allows for artistic creation of plausible novel
material appearances, bypassing the difficulty of acquiring novel samples. We
first synthesize novel materials, extending the existing MERL dataset up to 400
mathematically valid BRDFs. We then design a large-scale experiment, gathering
56,000 subjective ratings on the high-level perceptual attributes that best
describe our extended dataset of materials. Using these ratings, we build and
train networks of radial basis functions to act as functionals mapping the
perceptual attributes to an underlying PCA-based representation of BRDFs. We
show that our functionals are excellent predictors of the perceived attributes
of appearance. Our control space enables many applications, including intuitive
material editing of a wide range of visual properties, guidance for gamut
mapping, analysis of the correlation between perceptual attributes, or novel
appearance similarity metrics. Moreover, our methodology can be used to derive
functionals applicable to classic analytic BRDF representations. We release our
code and dataset publicly, in order to support and encourage further research
in this direction
An intuitive control space for material appearance
Many different techniques for measuring material appearance have been proposed in the last few years. These have produced large public datasets, which have been used for accurate, data-driven appearance modeling. However, although these datasets have allowed us to reach an unprecedented level of realism in visual appearance, editing the captured data remains a challenge. In this paper, we present an intuitive control space for predictable editing of captured BRDF data, which allows for artistic creation of plausible novel material appearances, bypassing the difficulty of acquiring novel samples. We first synthesize novel materials, extending the existing MERL dataset up to 400 mathematically valid BRDFs. We then design a large-scale experiment, gathering 56,000 subjective ratings on the high-level perceptual attributes that best describe our extended dataset of materials. Using these ratings, we build and train networks of radial basis functions to act as functionals mapping the perceptual attributes to an underlying PCA-based representation of BRDFs. We show that our functionals are excellent predictors of the perceived attributes of appearance. Our control space enables many applications, including intuitive material editing of a wide range of visual properties, guidance for gamut mapping, analysis of the correlation between perceptual attributes, or novel appearance similarity metrics. Moreover, our methodology can be used to derive functionals applicable to classic analytic BRDF representations. We release our code and dataset publicly, in order to support and encourage further research in this direction
Learning icons appearance similarity
Selecting an optimal set of icons is a crucial step in the pipeline of visual
design to structure and navigate through content. However, designing the icons
sets is usually a difficult task for which expert knowledge is required. In
this work, to ease the process of icon set selection to the users, we propose a
similarity metric which captures the properties of style and visual identity.
We train a Siamese Neural Network with an online dataset of icons organized in
visually coherent collections that are used to adaptively sample training data
and optimize the training process. As the dataset contains noise, we further
collect human-rated information on the perception of icon's similarity which
will be used for evaluating and testing the proposed model. We present several
results and applications based on searches, kernel visualizations and optimized
set proposals that can be helpful for designers and non-expert users while
exploring large collections of icons.Comment: 12 pages, 11 figure
How Will It Drape Like? Capturing Fabric Mechanics from Depth Images
We propose a method to estimate the mechanical parameters of fabrics using a
casual capture setup with a depth camera. Our approach enables to create
mechanically-correct digital representations of real-world textile materials,
which is a fundamental step for many interactive design and engineering
applications. As opposed to existing capture methods, which typically require
expensive setups, video sequences, or manual intervention, our solution can
capture at scale, is agnostic to the optical appearance of the textile, and
facilitates fabric arrangement by non-expert operators. To this end, we propose
a sim-to-real strategy to train a learning-based framework that can take as
input one or multiple images and outputs a full set of mechanical parameters.
Thanks to carefully designed data augmentation and transfer learning protocols,
our solution generalizes to real images despite being trained only on synthetic
data, hence successfully closing the sim-to-real loop.Key in our work is to
demonstrate that evaluating the regression accuracy based on the similarity at
parameter space leads to an inaccurate distances that do not match the human
perception. To overcome this, we propose a novel metric for fabric drape
similarity that operates on the image domain instead on the parameter space,
allowing us to evaluate our estimation within the context of a similarity rank.
We show that out metric correlates with human judgments about the perception of
drape similarity, and that our model predictions produce perceptually accurate
results compared to the ground truth parameters.Comment: 12 pages, 12 figures. Accepted to EUROGRAPHICS 2023. Project website:
https://carlosrodriguezpardo.es/projects/MechFromDepth