Color Recommendation for Vector Graphic Documents based on Multi-Palette Representation

Vector graphic documents present multiple visual elements, such as images, shapes, and texts. Choosing appropriate colors for multiple visual elements is a difficult but crucial task for both amateurs and professional designers. Instead of creating a single color palette for all elements, we extract multiple color palettes from each visual element in a graphic document, and then combine them into a color sequence. We propose a masked color model for color sequence completion and recommend the specified colors based on color context in multi-palette with high probability. We train the model and build a color recommendation system on a large-scale dataset of vector graphic documents. The proposed color recommendation method outperformed other state-of-the-art methods by both quantitative and qualitative evaluations on color prediction and our color recommendation system received positive feedback from professional designers in an interview study.Comment: Accepted to WACV 202

Reducing Ambiguities in Line-based Density Plots by Image-space Colorization

Line-based density plots are used to reduce visual clutter in line charts with a multitude of individual lines. However, these traditional density plots are often perceived ambiguously, which obstructs the user's identification of underlying trends in complex datasets. Thus, we propose a novel image space coloring method for line-based density plots that enhances their interpretability. Our method employs color not only to visually communicate data density but also to highlight similar regions in the plot, allowing users to identify and distinguish trends easily. We achieve this by performing hierarchical clustering based on the lines passing through each region and mapping the identified clusters to the hue circle using circular MDS. Additionally, we propose a heuristic approach to assign each line to the most probable cluster, enabling users to analyze density and individual lines. We motivate our method by conducting a small-scale user study, demonstrating the effectiveness of our method using synthetic and real-world datasets, and providing an interactive online tool for generating colored line-based density plots

Example-based Image Recoloring in Indoor Environment

Color structure of a home scene image closely relates to the material properties of its local regions. Existing color migration methods typically fail to fully infer the correlation between the coloring of local home scene regions, leading to a local blur problem. In this paper, we propose a color migration framework for home scene images. It picks the coloring from a template image and transforms such coloring to a home scene image through a simple interaction. Our framework comprises three main parts. First, we carry out an interactive segmentation to divide an image into local regions and extract their corresponding colors. Second, we generate a matching color table by sampling the template image according to the color structure of the original home scene image. Finally, we transform colors from the matching color table to the target home scene image with the boundary transition maintained. Experimental results show that our method can effectively transform the coloring of a scene matching with the color composition of a given natural or interior scenery

Graph Spectral Image Processing

Recent advent of graph signal processing (GSP) has spurred intensive studies of signals that live naturally on irregular data kernels described by graphs (e.g., social networks, wireless sensor networks). Though a digital image contains pixels that reside on a regularly sampled 2D grid, if one can design an appropriate underlying graph connecting pixels with weights that reflect the image structure, then one can interpret the image (or image patch) as a signal on a graph, and apply GSP tools for processing and analysis of the signal in graph spectral domain. In this article, we overview recent graph spectral techniques in GSP specifically for image / video processing. The topics covered include image compression, image restoration, image filtering and image segmentation

AutoPoster: A Highly Automatic and Content-aware Design System for Advertising Poster Generation

Advertising posters, a form of information presentation, combine visual and linguistic modalities. Creating a poster involves multiple steps and necessitates design experience and creativity. This paper introduces AutoPoster, a highly automatic and content-aware system for generating advertising posters. With only product images and titles as inputs, AutoPoster can automatically produce posters of varying sizes through four key stages: image cleaning and retargeting, layout generation, tagline generation, and style attribute prediction. To ensure visual harmony of posters, two content-aware models are incorporated for layout and tagline generation. Moreover, we propose a novel multi-task Style Attribute Predictor (SAP) to jointly predict visual style attributes. Meanwhile, to our knowledge, we propose the first poster generation dataset that includes visual attribute annotations for over 76k posters. Qualitative and quantitative outcomes from user studies and experiments substantiate the efficacy of our system and the aesthetic superiority of the generated posters compared to other poster generation methods.Comment: Accepted for ACM MM 202

A Survey for Graphic Design Intelligence

Graphic design is an effective language for visual communication. Using complex composition of visual elements (e.g., shape, color, font) guided by design principles and aesthetics, design helps produce more visually-appealing content. The creation of a harmonious design requires carefully selecting and combining different visual elements, which can be challenging and time-consuming. To expedite the design process, emerging AI techniques have been proposed to automatize tedious tasks and facilitate human creativity. However, most current works only focus on specific tasks targeting at different scenarios without a high-level abstraction. This paper aims to provide a systematic overview of graphic design intelligence and summarize literature in the taxonomy of representation, understanding and generation. Specifically we consider related works for individual visual elements as well as the overall design composition. Furthermore, we highlight some of the potential directions for future explorations.Comment: 10 pages, 2 figure

Engineering the Keratinocyte Microenvironment: Harnessing Topography to Direct Cellular Function

Skin wound healing presents a challenging and expensive clinical problem with nearly 20 million wounds requiring intervention leading to an annual cost of more than $8 million. Tissue engineered skin substitutes are valuable not only as a clinical therapy for chronic wounds and severe traumas, but also as in vitro 3D model systems to investigate wound healing and skin pathogenesis. However, these substitutes are limited by a lack of topography at the dermal-epidermal junction (DEJ). In contrast, the native DEJ is characterized by a series of dermal papillae which project upward into the epidermal layer and create physical topographic microniches that support keratinocyte stem cell clustering. In this thesis, we created novel 3D skin model systems to investigate the role of microtopography in regulating keratinocyte function and cell fate using scaffolds containing precisely engineered topographic features. We hypothesized that the microtopography of the DEJ creates distinct keratinocyte microniches that promote epidermal morphogenesis and modulate keratinocyte stem cell clustering which can be harnessed to create a more robust skin substitute that expedites wound closure. Using photolithographic techniques, we created micropatterned DEJ analogs and micropatterned dermal-epidermal regeneration matrices (µDERM) which couple a dermal support matrix to a micropatterned DEJ analog. We found that the incorporation of microtopography into our in vitro skin model resulted in a thicker, more robust epidermal layer. Additionally, we identified three distinct functional keratinocyte niches: the proliferative niche in narrow channels, the synthetic niche in wide channels and the keratinocyte stem cell niche in narrow channels and corner topographies. Ultimately, incorporation of both narrow and wide channels on a single construct allowed us to recreate native keratinocyte stem cell patterning in vitro. These model systems will allow us to investigate the role of cellular microniches in regulating cellular function and epidermal disease pathogenesis as well as to identify topographic cues that enhance the rate of wound healing

3D compositional hierarchies for object categorization

Deep learning methods have become the default tool for image classification. However, application of deep learning to surface shape classification is burdened by the limitations of existing methods, in particular, by lack of invariance to geometric transformations of input data. This thesis proposes two novel frameworks for learning a multi-layer representation of surface shape features, namely the view-based and the surface-based compositional hierarchical frameworks. The proposed representation is a hierarchical vocabulary of shape features, termed parts. Parts of the first layer are pre-defined, while parts of the subsequent layers, describing spatial relations of subparts, are learned. The view-based framework describes spatial relations between subparts using a camera-based reference frame. The key stage of the learning algorithm is part selection which forms the vocabulary based on multi-objective optimization, considering different importance measures of parts. Our experiments show that this framework enables efficient category recognition on a large-scale dataset. The surface-based framework exploits part-based intrinsic reference frames, which are computed for lower layers parts and inherited by parts of the subsequent layers. During learning spatial relations between subparts are described in these reference frames. During inference, a part is detected in input data when its subparts are detected at certain positions and orientations in each other’s reference frames. Since rigid body transformations don’t change positions and orientations of parts in intrinsic reference frames, this approach enables efficient recognition from unseen poses. Experiments show that this framework exhibits a large discriminative power and greater robustness to rigid body transformations than advanced CNN-based methods