2,343 research outputs found
Text-Guided Scene Sketch-to-Photo Synthesis
We propose a method for scene-level sketch-to-photo synthesis with text
guidance. Although object-level sketch-to-photo synthesis has been widely
studied, whole-scene synthesis is still challenging without reference photos
that adequately reflect the target style. To this end, we leverage knowledge
from recent large-scale pre-trained generative models, resulting in text-guided
sketch-to-photo synthesis without the need for reference images. To train our
model, we use self-supervised learning from a set of photographs. Specifically,
we use a pre-trained edge detector that maps both color and sketch images into
a standardized edge domain, which reduces the gap between photograph-based edge
images (during training) and hand-drawn sketch images (during inference). We
implement our method by fine-tuning a latent diffusion model (i.e., Stable
Diffusion) with sketch and text conditions. Experiments show that the proposed
method translates original sketch images that are not extracted from color
images into photos with compelling visual quality
Validation of an Aesthetic Assessment System for Commercial Tasks
[Abstract] Automatic prediction of the aesthetic value of images has received increasing attention in recent years. This is due, on the one hand, to the potential impact that predicting the aesthetic value has on practical applications. Even so, it remains a difficult task given the subjectivity and complexity of the problem. An image aesthetics assessment system was developed in recent years by our research group. In this work, its potential to be applied in commercial tasks is tested. With this objective, a set of three portals and three real estate agencies in Spain were taken as case studies. Images of their websites were taken to build the experimental dataset and a validation method was developed to test their original order with another proposed one according to their aesthetic value. So, in this new order, the images that have the high aesthetic score by the AI system will occupy the first positions of the portal. Relevant results were obtained, with an average increase of 52.54% in the number of clicks on the ads, in the experiment with Real Estate portals. A statistical analysis prove that there is a significant difference in the number of clicks after selecting the images with the AI system.This work is supported by the General Directorate of Culture, Education and University Management of Xunta de Galicia (Ref. ED431D 201716), Competitive Reference Groups (Ref. ED431C 201849) and Ministry of Science and Innovation project Society challenges (Ref. PID2020-118362RB-I00). We also wish to acknowledge the support received from the Centro de Investigación de Galicia “CITIC”, funded by Xunta de Galicia and the European Union (European Regional Development Fund- Galicia 2014-2020 Program), by grant ED431G 2019/01Xunta de Galicia; ED431D 201716Xunta de Galicia; ED431C 201849Xunta de Galicia; ED431G 2019/0
Puffy: A Step-by-step Guide to Craft Bio-inspired Artifacts with Interactive Materiality
A rising number of HCI scholars have begun to use materiality as a starting
point for exploring the design's potential and restrictions. Despite the
theoretical flourishing, the practical design process and instruction for
beginner practitioners are still in scarcity. We leveraged the pictorial format
to illustrate our crafting process of Puffy, a bio-inspired artifact that
features a cilia-mimetic surface expressing anthropomorphic qualities through
shape changes. Our approach consists of three key activities (i.e., analysis,
synthesis, and detailing) interlaced recursively throughout the journey. Using
this approach, we analyzed different input sources, synthesized peers'
critiques and self-reflection, and detailed the designed experience with
iterative prototypes. Building on a reflective analysis of our approach, we
concluded with a set of practical implications and design recommendations to
inform other practitioners to initiate their investigations in interactive
materiality.Comment: 17th International Conference On Tangible Embedded And Embodied
Interactio
Enhancing travel experience with the combination of information visualization, situation awareness, and distributed cognition
With the new forms of travel introduced by new technologies of transportation and communication, a satisfied travel experience could be affected by various factors before and during a trip. Especially for road trips, traveling by car provides freedom on time control while leading to more possibilities of rescheduling initial plans made under time constraints. When overwhelmed with the need for changed travel context to avoid unexpected events that will require a serious change of initial plans, travelers need to find and access helpful contextual information quickly. This is a context-related decision making process that requires amplifying human situation awareness and supporting distributed cognition, since travel information offers multiple choices. To solve this problem, I applied information visualization as the main design solution. When comparing it with a traditional representation of lists, information visualization displays the advantages of visual representation of abstract data to clarify and depict the information and amplify cognition while improving travel experience intuitively in the domain of user experience design. Therefore in this thesis I will address the approach of implementing recontextualized situation awareness, distributed cognition, and information visualization in a travel-aid system. By using both theoretical and practical design perspectives, I will discuss how to enhance travel experience with represented contextual information that users desire or expect before and during a road trip. I will also explore the new values of this design with strategic business support. Additionally, after conducting research and analysis on existing interaction design parts, I selected a smartphone app to serve as a proper platform with connected multifunctions. Briefly, I begin the thesis with a review of previous theories and aspects of travel planning, information visualization as it relates to travel, situation awareness, and distributed cognition in the design context and related smartphone apps. Then I discuss the process of identifying the specific issues to be solved or improved with a preliminary research of empirical study, followed by an interview, online survey, insights synthesis, and business model design. After a visual-system design was developed, heuristic evaluation was employed to assess the outcome. Lastly, a new round of refined design results is introduced based on outcomes of the evaluation
SimpSON: Simplifying Photo Cleanup with Single-Click Distracting Object Segmentation Network
In photo editing, it is common practice to remove visual distractions to
improve the overall image quality and highlight the primary subject. However,
manually selecting and removing these small and dense distracting regions can
be a laborious and time-consuming task. In this paper, we propose an
interactive distractor selection method that is optimized to achieve the task
with just a single click. Our method surpasses the precision and recall
achieved by the traditional method of running panoptic segmentation and then
selecting the segments containing the clicks. We also showcase how a
transformer-based module can be used to identify more distracting regions
similar to the user's click position. Our experiments demonstrate that the
model can effectively and accurately segment unknown distracting objects
interactively and in groups. By significantly simplifying the photo cleaning
and retouching process, our proposed model provides inspiration for exploring
rare object segmentation and group selection with a single click.Comment: CVPR 2023. Project link: https://simpson-cvpr23.github.i
Progressive Joint Low-light Enhancement and Noise Removal for Raw Images
Low-light imaging on mobile devices is typically challenging due to
insufficient incident light coming through the relatively small aperture,
resulting in a low signal-to-noise ratio. Most of the previous works on
low-light image processing focus either only on a single task such as
illumination adjustment, color enhancement, or noise removal; or on a joint
illumination adjustment and denoising task that heavily relies on short-long
exposure image pairs collected from specific camera models, and thus these
approaches are less practical and generalizable in real-world settings where
camera-specific joint enhancement and restoration is required. To tackle this
problem, in this paper, we propose a low-light image processing framework that
performs joint illumination adjustment, color enhancement, and denoising.
Considering the difficulty in model-specific data collection and the ultra-high
definition of the captured images, we design two branches: a coefficient
estimation branch as well as a joint enhancement and denoising branch. The
coefficient estimation branch works in a low-resolution space and predicts the
coefficients for enhancement via bilateral learning, whereas the joint
enhancement and denoising branch works in a full-resolution space and
progressively performs joint enhancement and denoising. In contrast to existing
methods, our framework does not need to recollect massive data when being
adapted to another camera model, which significantly reduces the efforts
required to fine-tune our approach for practical usage. Through extensive
experiments, we demonstrate its great potential in real-world low-light imaging
applications when compared with current state-of-the-art methods
- …