1,283 research outputs found
Calipso: Physics-based Image and Video Editing through CAD Model Proxies
We present Calipso, an interactive method for editing images and videos in a
physically-coherent manner. Our main idea is to realize physics-based
manipulations by running a full physics simulation on proxy geometries given by
non-rigidly aligned CAD models. Running these simulations allows us to apply
new, unseen forces to move or deform selected objects, change physical
parameters such as mass or elasticity, or even add entire new objects that
interact with the rest of the underlying scene. In Calipso, the user makes
edits directly in 3D; these edits are processed by the simulation and then
transfered to the target 2D content using shape-to-image correspondences in a
photo-realistic rendering process. To align the CAD models, we introduce an
efficient CAD-to-image alignment procedure that jointly minimizes for rigid and
non-rigid alignment while preserving the high-level structure of the input
shape. Moreover, the user can choose to exploit image flow to estimate scene
motion, producing coherent physical behavior with ambient dynamics. We
demonstrate Calipso's physics-based editing on a wide range of examples
producing myriad physical behavior while preserving geometric and visual
consistency.Comment: 11 page
A robust patch-based synthesis framework for combining inconsistent images
Current methods for combining different images produce visible artifacts when the sources have very different textures and structures, come from far view points, or capture dynamic scenes with motions. In this thesis, we propose a patch-based synthesis algorithm to plausibly combine different images that have color, texture, structural, and geometric inconsistencies. For some applications such as cloning and stitching where a gradual blend is required, we present a new method for synthesizing a transition region between two source images, such that inconsistent properties change gradually from one source to the other. We call this process image melding. For gradual blending, we generalized patch-based optimization foundation with three key generalizations: First, we enrich the patch search space with additional geometric and photometric transformations. Second, we integrate image gradients into the patch representation and replace the usual color averaging with a screened Poisson equation solver. Third, we propose a new energy based on mixed L2/L0 norms for colors and gradients that produces a gradual transition between sources without sacrificing texture sharpness. Together, all three generalizations enable patch-based solutions to a broad class of image melding problems involving inconsistent sources: object cloning, stitching challenging panoramas, hole filling from multiple photos, and image harmonization. We also demonstrate another application which requires us to address inconsistencies across the images: high dynamic range (HDR) reconstruction using sequential exposures. In this application, the results will suffer from objectionable artifacts for dynamic scenes if the inconsistencies caused by significant scene motions are not handled properly. In this thesis, we propose a new approach to HDR reconstruction that uses information in all exposures while being more robust to motion than previous techniques. Our algorithm is based on a novel patch-based energy-minimization formulation that integrates alignment and reconstruction in a joint optimization through an equation we call the HDR image synthesis equation. This allows us to produce an HDR result that is aligned to one of the exposures yet contains information from all of them. These two applications (image melding and high dynamic range reconstruction) show that patch based methods like the one proposed in this dissertation can address inconsistent images and could open the door to many new image editing applications in the future
A method for three-dimensional reconstruction of a train accident scene using photographs
Railway accidents that usually cause numerous property and life losses occurred in recent years all around the world. In general, resources such as financial supports and incident rescue programs are required to minimize the losses after an accident. Due to lack of comprehensive information collected from accident sites, most railway emergency management departments face a predicament in setting up rescue schemes. To tackle the issue, realistic three-dimensional virtual accident scene reconstruction technology is developed, which provides and visualises supplementary materials and information about a train accident and can offer assistance to emergency crews when making decisions. We propose a photo-based three-dimensional reconstruction framework of vehicles for measuring the positions and poses of carriages involved in an accident. We implement and examine two case studies to validate this reconstruction method, which performs well in the assigned tasks
Ameliorating the Saliency of Objects in digital Images
Visual saliency is the capability of a revelation system of human or machine to choose a sub part of information for further processing. The mechanism describes here serves as a filter to pick only the interesting information related to given errands and the tasks while ignoring inappropriate information. This work uses Random Forest to know the similarity between the image patches, apply active contour model to get the approximate contour and do dynamic thresholding segmentation .The results we get consists of many small artifacts, so to remove the low level details and to obtain the more smoothness we apply gradient minimization technique
CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulation
Realistic simulation is key to enabling safe and scalable development of %
self-driving vehicles. A core component is simulating the sensors so that the
entire autonomy system can be tested in simulation. Sensor simulation involves
modeling traffic participants, such as vehicles, with high quality appearance
and articulated geometry, and rendering them in real time. The self-driving
industry has typically employed artists to build these assets. However, this is
expensive, slow, and may not reflect reality. Instead, reconstructing assets
automatically from sensor data collected in the wild would provide a better
path to generating a diverse and large set with good real-world coverage.
Nevertheless, current reconstruction approaches struggle on in-the-wild sensor
data, due to its sparsity and noise. To tackle these issues, we present CADSim,
which combines part-aware object-class priors via a small set of CAD models
with differentiable rendering to automatically reconstruct vehicle geometry,
including articulated wheels, with high-quality appearance. Our experiments
show our method recovers more accurate shapes from sparse data compared to
existing approaches. Importantly, it also trains and renders efficiently. We
demonstrate our reconstructed vehicles in several applications, including
accurate testing of autonomy perception systems.Comment: CoRL 2022. Project page: https://waabi.ai/cadsim
- …