62 research outputs found
Sparse graph regularized mesh color edit propagation
Mesh color edit propagation aims to propagate the color from a few color strokes to the whole mesh, which is useful for mesh colorization, color enhancement and color editing, etc. Compared with image edit propagation, luminance information is not available for 3D mesh data, so the color edit propagation is more difficult on 3D meshes than images, with far less research carried out. This paper proposes a novel solution based on sparse graph regularization. Firstly, a few color strokes are interactively drawn by the user, and then the color will be propagated to the whole mesh by minimizing a sparse graph regularized nonlinear energy function. The proposed method effectively measures geometric similarity over shapes by using a set of complementary multiscale feature descriptors, and effectively controls color bleeding via a sparse
â„“
1
optimization rather than quadratic minimization used in existing work. The proposed framework can be applied for the task of interactive mesh colorization, mesh color enhancement and mesh color editing. Extensive qualitative and quantitative experiments show that the proposed method outperforms the state-of-the-art methods
Automatic example-based image colorization using location-aware cross-scale matching
Given a reference colour image and a destination grayscale image, this paper presents a novel automatic colourisation algorithm that transfers colour information from the reference image to the destination image. Since the reference and destination images may contain content at different or even varying scales (due to changes of distance between objects and the camera), existing texture matching based methods can often perform poorly. We propose a novel cross-scale texture matching method to improve the robustness and quality of the colourisation results. Suitable matching scales are considered locally, which are then fused using global optimisation that minimises both the matching errors and spatial change of scales. The minimisation is efficiently solved using a multi-label graph-cut algorithm. Since only low-level texture features are used, texture matching based colourisation can still produce semantically incorrect results, such as meadow appearing above the sky. We consider a class of semantic violation where the statistics of up-down relationships learnt from the reference image are violated and propose an effective method to identify and correct unreasonable colourisation. Finally, a novel nonlocal â„“1 optimisation framework is developed to propagate high confidence micro-scribbles to regions of lower confidence to produce a fully colourised image. Qualitative and quantitative evaluations show that our method outperforms several state-of-the-art methods
Scribble-based gradient mesh recoloring
Previous gradient mesh recoloring methods usually have dependencies on an additional reference image and the rasterized gradient mesh. To circumvent such dependencies, we propose a user scribble-based recoloring method, in which users are allowed to annotate gradient meshes with a few color scribbles. Our approach builds an auxiliary mesh from gradient meshes, namely control net, by taking both colors and local color gradients at mesh points into account. We then develop an extended chrominance blending method to propagate the user specified colors over the control net. The recolored gradient mesh is finally reconstructed from the recolored control net. Experiments validate the effectiveness of our approach on multiple gradient meshes. Compared with various alternative solutions, our method has no color bleedings nor sampling artifacts, and can achieve fast performance
Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining
Single image rain streaks removal has recently witnessed substantial progress
due to the development of deep convolutional neural networks. However, existing
deep learning based methods either focus on the entrance and exit of the
network by decomposing the input image into high and low frequency information
and employing residual learning to reduce the mapping range, or focus on the
introduction of cascaded learning scheme to decompose the task of rain streaks
removal into multi-stages. These methods treat the convolutional neural network
as an encapsulated end-to-end mapping module without deepening into the
rationality and superiority of neural network design. In this paper, we delve
into an effective end-to-end neural network structure for stronger feature
expression and spatial correlation learning. Specifically, we propose a
non-locally enhanced encoder-decoder network framework, which consists of a
pooling indices embedded encoder-decoder network to efficiently learn
increasingly abstract feature representation for more accurate rain streaks
modeling while perfectly preserving the image detail. The proposed
encoder-decoder framework is composed of a series of non-locally enhanced dense
blocks that are designed to not only fully exploit hierarchical features from
all the convolutional layers but also well capture the long-distance
dependencies and structural information. Extensive experiments on synthetic and
real datasets demonstrate that the proposed method can effectively remove
rain-streaks on rainy image of various densities while well preserving the
image details, which achieves significant improvements over the recent
state-of-the-art methods.Comment: Accepted to ACM Multimedia 201
SVCNet: Scribble-based Video Colorization Network with Temporal Aggregation
In this paper, we propose a scribble-based video colorization network with
temporal aggregation called SVCNet. It can colorize monochrome videos based on
different user-given color scribbles. It addresses three common issues in the
scribble-based video colorization area: colorization vividness, temporal
consistency, and color bleeding. To improve the colorization quality and
strengthen the temporal consistency, we adopt two sequential sub-networks in
SVCNet for precise colorization and temporal smoothing, respectively. The first
stage includes a pyramid feature encoder to incorporate color scribbles with a
grayscale frame, and a semantic feature encoder to extract semantics. The
second stage finetunes the output from the first stage by aggregating the
information of neighboring colorized frames (as short-range connections) and
the first colorized frame (as a long-range connection). To alleviate the color
bleeding artifacts, we learn video colorization and segmentation
simultaneously. Furthermore, we set the majority of operations on a fixed small
image resolution and use a Super-resolution Module at the tail of SVCNet to
recover original sizes. It allows the SVCNet to fit different image resolutions
at the inference. Finally, we evaluate the proposed SVCNet on DAVIS and Videvo
benchmarks. The experimental results demonstrate that SVCNet produces both
higher-quality and more temporally consistent videos than other well-known
video colorization approaches. The codes and models can be found at
https://github.com/zhaoyuzhi/SVCNet.Comment: accepted by IEEE Transactions on Image Processing (TIP
- …