336 research outputs found
Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision
Denoising diffusion models are a powerful type of generative models used to
capture complex distributions of real-world signals. However, their
applicability is limited to scenarios where training samples are readily
available, which is not always the case in real-world applications. For
example, in inverse graphics, the goal is to generate samples from a
distribution of 3D scenes that align with a given image, but ground-truth 3D
scenes are unavailable and only 2D images are accessible. To address this
limitation, we propose a novel class of denoising diffusion probabilistic
models that learn to sample from distributions of signals that are never
directly observed. Instead, these signals are measured indirectly through a
known differentiable forward model, which produces partial observations of the
unknown signal. Our approach involves integrating the forward model directly
into the denoising process. This integration effectively connects the
generative modeling of observations with the generative modeling of the
underlying signals, allowing for end-to-end training of a conditional
generative model over signals. During inference, our approach enables sampling
from the distribution of underlying signals that are consistent with a given
partial observation. We demonstrate the effectiveness of our method on three
challenging computer vision tasks. For instance, in the context of inverse
graphics, our model enables direct sampling from the distribution of 3D scenes
that align with a single 2D input image.Comment: Project page: https://diffusion-with-forward-models.github.io
Seeing 3D Objects in a Single Image via Self-Supervised Static-Dynamic Disentanglement
Human perception reliably identifies movable and immovable parts of 3D
scenes, and completes the 3D structure of objects and background from
incomplete observations. We learn this skill not via labeled examples, but
simply by observing objects move. In this work, we propose an approach that
observes unlabeled multi-view videos at training time and learns to map a
single image observation of a complex scene, such as a street with cars, to a
3D neural scene representation that is disentangled into movable and immovable
parts while plausibly completing its 3D structure. We separately parameterize
movable and immovable scene parts via 2D neural ground plans. These ground
plans are 2D grids of features aligned with the ground plane that can be
locally decoded into 3D neural radiance fields. Our model is trained
self-supervised via neural rendering. We demonstrate that the structure
inherent to our disentangled 3D representation enables a variety of downstream
tasks in street-scale 3D scenes using simple heuristics, such as extraction of
object-centric 3D representations, novel view synthesis, instance segmentation,
and 3D bounding box prediction, highlighting its value as a backbone for
data-efficient 3D scene understanding models. This disentanglement further
enables scene editing via object manipulation such as deletion, insertion, and
rigid-body motion.Comment: Project page: https://prafullsharma.net/see3d
The influence of task difficulty on engagement, performance and self-efficacy
peer-reviewedMy research examined the impact of a person’s belief about their own capabilities and how
this influences their performance. In order to examine this I needed a task that was both
relatively enjoyable, so that participants would engage with it in their own free time without
pressure to do so, and a task that was not heavily linked to a particular subject as this would
influence performance. That is the line of thinking that led to a PhD examining self-efficacy
theory by getting hundreds of children to play Pacman, a popular arcade gameACCEPTEDPeer reviewe
State of the Art on Neural Rendering
Efficient rendering of photo-realistic virtual worlds is a long standing effort of computer graphics. Modern graphics techniques have succeeded in synthesizing photo-realistic images from hand-crafted scene representations. However, the automatic generation of shape, materials, lighting, and other aspects of scenes remains a challenging problem that, if solved, would make photo-realistic computer graphics more widely accessible. Concurrently, progress in computer vision and machine learning have given rise to a new approach to image synthesis and editing, namely deep generative models. Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training. With a plethora of applications in computer graphics and vision, neural rendering is poised to become a new area in the graphics community, yet no survey of this emerging field exists. This state-of-the-art report summarizes the recent trends and applications of neural rendering. We focus on approaches that combine classic computer graphics techniques with deep generative models to obtain controllable and photo-realistic outputs. Starting with an overview of the underlying computer graphics and machine learning concepts, we discuss critical aspects of neural rendering approaches. This state-of-the-art report is focused on the many important use cases for the described algorithms such as novel view synthesis, semantic photo manipulation, facial and body reenactment, relighting, free-viewpoint video, and the creation of photo-realistic avatars for virtual and augmented reality telepresence. Finally, we conclude with a discussion of the social implications of such technology and investigate open research problems
Comparative effectiveness of a serious game and an e-module to support patient safety knowledge and awareness
Promises and Prospects of Educational Technology, Evidence from Systematic Reviews and Meta-analyses
Lessons Learned from Creating a Mobile Version of an Educational Board Game to Increase Situational Awareness
This paper reports on an iterative design process for a serious game, which aims to raise situational awareness among different stakeholders in a logistics value chain by introducing multi-user role-playing games. It does so in several phases: After introducing the field of logistics as a problem domain for an educational challenge, it firstly describes the design of an educational board game for the field of disruption handling in logistics processes. Secondly, it de-scribes how the board game can be realized in an open-source mobile serious games platform and identifies lessons learned based on advantages and issues found. Thirdly, it derives requirements for a re-design of the mobile game and finally draws conclusions.SALOM
Challenges in QCD matter physics - The Compressed Baryonic Matter experiment at FAIR
Substantial experimental and theoretical efforts worldwide are devoted to
explore the phase diagram of strongly interacting matter. At LHC and top RHIC
energies, QCD matter is studied at very high temperatures and nearly vanishing
net-baryon densities. There is evidence that a Quark-Gluon-Plasma (QGP) was
created at experiments at RHIC and LHC. The transition from the QGP back to the
hadron gas is found to be a smooth cross over. For larger net-baryon densities
and lower temperatures, it is expected that the QCD phase diagram exhibits a
rich structure, such as a first-order phase transition between hadronic and
partonic matter which terminates in a critical point, or exotic phases like
quarkyonic matter. The discovery of these landmarks would be a breakthrough in
our understanding of the strong interaction and is therefore in the focus of
various high-energy heavy-ion research programs. The Compressed Baryonic Matter
(CBM) experiment at FAIR will play a unique role in the exploration of the QCD
phase diagram in the region of high net-baryon densities, because it is
designed to run at unprecedented interaction rates. High-rate operation is the
key prerequisite for high-precision measurements of multi-differential
observables and of rare diagnostic probes which are sensitive to the dense
phase of the nuclear fireball. The goal of the CBM experiment at SIS100
(sqrt(s_NN) = 2.7 - 4.9 GeV) is to discover fundamental properties of QCD
matter: the phase structure at large baryon-chemical potentials (mu_B > 500
MeV), effects of chiral symmetry, and the equation-of-state at high density as
it is expected to occur in the core of neutron stars. In this article, we
review the motivation for and the physics programme of CBM, including
activities before the start of data taking in 2022, in the context of the
worldwide efforts to explore high-density QCD matter.Comment: 15 pages, 11 figures. Published in European Physical Journal
- …