4 research outputs found
Deep Video Color Propagation
Traditional approaches for color propagation in videos rely on some form of
matching between consecutive video frames. Using appearance descriptors, colors
are then propagated both spatially and temporally. These methods, however, are
computationally expensive and do not take advantage of semantic information of
the scene. In this work we propose a deep learning framework for color
propagation that combines a local strategy, to propagate colors frame-by-frame
ensuring temporal stability, and a global strategy, using semantics for color
propagation within a longer range. Our evaluation shows the superiority of our
strategy over existing video and image color propagation methods as well as
neural photo-realistic style transfer approaches.Comment: BMVC 201
Exemplar-based Video Colorization with Long-term Spatiotemporal Dependency
Exemplar-based video colorization is an essential technique for applications
like old movie restoration. Although recent methods perform well in still
scenes or scenes with regular movement, they always lack robustness in moving
scenes due to their weak ability in modeling long-term dependency both
spatially and temporally, leading to color fading, color discontinuity or other
artifacts. To solve this problem, we propose an exemplar-based video
colorization framework with long-term spatiotemporal dependency. To enhance the
long-term spatial dependency, a parallelized CNN-Transformer block and a double
head non-local operation are designed. The proposed CNN-Transformer block can
better incorporate long-term spatial dependency with local texture and
structural features, and the double head non-local operation further leverages
the performance of augmented feature. While for long-term temporal dependency
enhancement, we further introduce the novel linkage subnet. The linkage subnet
propagate motion information across adjacent frame blocks and help to maintain
temporal continuity. Experiments demonstrate that our model outperforms recent
state-of-the-art methods both quantitatively and qualitatively. Also, our model
can generate more colorful, realistic and stabilized results, especially for
scenes where objects change greatly and irregularly
Recent Advances in Signal Processing
The signal processing task is a very critical issue in the majority of new technological inventions and challenges in a variety of applications in both science and engineering fields. Classical signal processing techniques have largely worked with mathematical models that are linear, local, stationary, and Gaussian. They have always favored closed-form tractability over real-world accuracy. These constraints were imposed by the lack of powerful computing tools. During the last few decades, signal processing theories, developments, and applications have matured rapidly and now include tools from many areas of mathematics, computer science, physics, and engineering. This book is targeted primarily toward both students and researchers who want to be exposed to a wide variety of signal processing techniques and algorithms. It includes 27 chapters that can be categorized into five different areas depending on the application at hand. These five categories are ordered to address image processing, speech processing, communication systems, time-series analysis, and educational packages respectively. The book has the advantage of providing a collection of applications that are completely independent and self-contained; thus, the interested reader can choose any chapter and skip to another without losing continuity