384 research outputs found
SRU-NET: SOBEL RESIDUAL U-NET FOR IMAGE MANIPULATION DETECTION
Recently, most successful image manipulation detection methods have been based on convolutional neural networks (CNNs). Nevertheless, Existing CNN methods have limited abilities. CNN-based detection networks tend to extract signal features strongly related to content. However, image manipulation detection tends to extract weak signal features that are weakly related to content. To address this issue, We propose a novel Sobel residual neural network with adaptive central difference convolution, an extension of the classical U-Net architecture, for image manipulation detection. Adaptive central differential convolution can capture the essential attributes of an image by gathering intensity and gradient information. Sobel residual gradient block can capture forgery edge discriminative details. Extensive experimental results show that our method can significantly improve the accuracy of localising the forged region compared with the state-of-the-art methods
Controllable Motion Diffusion Model
Generating realistic and controllable motions for virtual characters is a
challenging task in computer animation, and its implications extend to games,
simulations, and virtual reality. Recent studies have drawn inspiration from
the success of diffusion models in image generation, demonstrating the
potential for addressing this task. However, the majority of these studies have
been limited to offline applications that target at sequence-level generation
that generates all steps simultaneously. To enable real-time motion synthesis
with diffusion models in response to time-varying control signals, we propose
the framework of the Controllable Motion Diffusion Model (COMODO). Our
framework begins with an auto-regressive motion diffusion model (A-MDM), which
generates motion sequences step by step. In this way, simply using the standard
DDPM algorithm without any additional complexity, our framework is able to
generate high-fidelity motion sequences over extended periods with different
types of control signals. Then, we propose our reinforcement learning-based
controller and controlling strategies on top of the A-MDM model, so that our
framework can steer the motion synthesis process across multiple tasks,
including target reaching, joystick-based control, goal-oriented control, and
trajectory following. The proposed framework enables the real-time generation
of diverse motions that react adaptively to user commands on-the-fly, thereby
enhancing the overall user experience. Besides, it is compatible with the
inpainting-based editing methods and can predict much more diverse motions
without additional fine-tuning of the basic motion generation models. We
conduct comprehensive experiments to evaluate the effectiveness of our
framework in performing various tasks and compare its performance against
state-of-the-art methods
Cinematic Behavior Transfer via NeRF-based Differentiable Filming
In the evolving landscape of digital media and video production, the precise
manipulation and reproduction of visual elements like camera movements and
character actions are highly desired. Existing SLAM methods face limitations in
dynamic scenes and human pose estimation often focuses on 2D projections,
neglecting 3D statuses. To address these issues, we first introduce a reverse
filming behavior estimation technique. It optimizes camera trajectories by
leveraging NeRF as a differentiable renderer and refining SMPL tracks. We then
introduce a cinematic transfer pipeline that is able to transfer various shot
types to a new 2D video or a 3D virtual environment. The incorporation of 3D
engine workflow enables superior rendering and control abilities, which also
achieves a higher rating in the user study.Comment: Project Page:
https://virtualfilmstudio.github.io/projects/cinetransfe
- …