Conditional Generation from Unconditional Diffusion Models using Denoiser Representations

Denoising diffusion models have gained popularity as a generative modeling technique for producing high-quality and diverse images. Applying these models to downstream tasks requires conditioning, which can take the form of text, class labels, or other forms of guidance. However, providing conditioning information to these models can be challenging, particularly when annotations are scarce or imprecise. In this paper, we propose adapting pre-trained unconditional diffusion models to new conditions using the learned internal representations of the denoiser network. We demonstrate the effectiveness of our approach on various conditional generation tasks, including attribute-conditioned generation and mask-conditioned generation. Additionally, we show that augmenting the Tiny ImageNet training set with synthetic images generated by our approach improves the classification accuracy of ResNet baselines by up to 8%. Our approach provides a powerful and flexible way to adapt diffusion models to new conditions and generate high-quality augmented data for various conditional generation tasks

S-VolSDF: Sparse Multi-View Stereo Regularization of Neural Implicit Surfaces

Neural rendering of implicit surfaces performs well in 3D vision applications. However, it requires dense input views as supervision. When only sparse input images are available, output quality drops significantly due to the shape-radiance ambiguity problem. We note that this ambiguity can be constrained when a 3D point is visible in multiple views, as is the case in multi-view stereo (MVS). We thus propose to regularize neural rendering optimization with an MVS solution. The use of an MVS probability volume and a generalized cross entropy loss leads to a noise-tolerant optimization process. In addition, neural rendering provides global consistency constraints that guide the MVS depth hypothesis sampling and thus improves MVS performance. Given only three sparse input views, experiments show that our method not only outperforms generic neural rendering models by a large margin but also significantly increases the reconstruction quality of MVS models. Project webpage: https://hao-yu-wu.github.io/s-volsdf/

Rotationally induced ingress in rotor–stator systems

The presence of a rotating disk adjacent to a stationary disk forms a rotor–stator cavity known as a wheel-space. It is necessary for gas turbine wheel-spaces to be purged with sealing flow bled from the compressor to counteract the harmful effects of ingress. This paper presents a combined experimental, theoretical, and computational study of rotationally induced ingress in rotor–stator systems. Measurements were made in a wheel-space with an axial clearance rim seal under axisymmetric conditions in the absence of a mainstream annulus through-flow. Ingress was quantified using a gas concentration technique and the flow structure in the cavity was explored with static and total pressure measurements to determine the swirl ratio. A low-order theoretical model was developed based on the boundary layer momentum-integral equations. The theory gave excellent results when predicting the effects of ingress and purge flows on the radial pressure and swirl gradients. Unsteady Reynolds-Averaged Navier–Stokes computations were conducted to provide greater fluid dynamic insight into the wheel-space flow structure and ingress through the rim seal. The computational results demonstrated some of the closest agreement with experimental measurements of ingress available in the literature, showing that rotationally induced ingress is dominated by unsteady large-scale structures in the rim seal gap instead of the previously ascribed disk-pumping effect. The study serves as an important validation case for investigations of ingress in rotor–stator systems in more complex environments

A New Interpretation of Hot Gas Ingress Through Turbine Rim Seals Influenced by Mainstream Annulus Swirl

Rim seals are fitted at the periphery of the stator and rotor disks to reduce the adverse effects of hot gas ingress on highly stressed turbine components limited by temperature. Ingress is induced by rotational effects such as disk pumping, as well as by asymmetric pressure-driven unsteady phenomena. These influences superpose to form a complex flow-physics problem that is a challenge for computational fluid dynamics. Engine designers typically use practical low-order models that require empirical validation and correlating parameters. This paper identifies the swirl ratio in the mainstream annulus as a dominant characterizing parameter to predict ingress. This is a new interpretation that is supported by extending a low-order model based on turbulent transport using an effective eddy mixing length based on the difference in swirl between the annulus and seal clearance. Experimental measurements were made using a 1.5-stage turbine rig at low Reynolds number. The influence of annulus swirl ratio was investigated over a range of flow conditions and two rim-seal geometries, with the ingress quantified using CO 2tracer concentration in the sealing flow. The concentration data were complemented by measurements in the annulus using a five-hole aerodynamic probe.</p

Rotationally induced ingress in rotor–stator systems

The presence of a rotating disk adjacent to a stationary disk forms a rotor–stator cavity known as a wheel-space. It is necessary for gas turbine wheel-spaces to be purged with sealing flow bled from the compressor to counteract the harmful effects of ingress. This paper presents a combined experimental, theoretical, and computational study of rotationally induced ingress in rotor–stator systems. Measurements were made in a wheel-space with an axial clearance rim seal under axisymmetric conditions in the absence of a mainstream annulus through-flow. Ingress was quantified using a gas concentration technique and the flow structure in the cavity was explored with static and total pressure measurements to determine the swirl ratio. A low-order theoretical model was developed based on the boundary layer momentum-integral equations. The theory gave excellent results when predicting the effects of ingress and purge flows on the radial pressure and swirl gradients. Unsteady Reynolds-Averaged Navier–Stokes computations were conducted to provide greater fluid dynamic insight into the wheel-space flow structure and ingress through the rim seal. The computational results demonstrated some of the closest agreement with experimental measurements of ingress available in the literature, showing that rotationally induced ingress is dominated by unsteady large-scale structures in the rim seal gap instead of the previously ascribed disk-pumping effect. The study serves as an important validation case for investigations of ingress in rotor–stator systems in more complex environments

Influence of Flow Coefficient on Ingress Through Turbine Rim Seals

Rim seals are critical in terms of limiting the temperature of highly-stressed engine components but function with a penalty to the power output and contribute to entropy gain stemming from mixing losses in the turbine. Ingress through rim seals is influenced by the presence of rotor blades and stator vanes, and the mainstream flow coefficient in the annulus that determines the corresponding swirl. This paper presents an experimental study of ingress upstream and downstream of the rotor disc in a 1.5-stage rig with double radial clearance rim seals. Two rotor discs were used, one with blades and one without, and two platforms were used downstream of the rotor, one with vanes and one without. Tests were conducted at two rotational speeds and a range of flow conditions was achieved by varying the annulus and sealing mass flow rates. Concentration effectiveness, swirl and steady pressure measurements separated, for the first time, the influence of the blades and vanes on ingressover a wide range of flow conditions. Measurements on the downstream stator platform provide added insight into the complex interaction between the egress and the mainstream.Measurements of unsteady pressure revealed the presence of large-scale structures, even in the absence of blades. The number and speed of the structures was shown to depend on the flow coefficient and the purge flow rate

GFlowNet-EM for learning compositional latent variable models

Latent variable models (LVMs) with discrete compositional latents are an important but challenging setting due to a combinatorially large number of possible configurations of the latents. A key tradeoff in modeling the posteriors over latents is between expressivity and tractable optimization. For algorithms based on expectation-maximization (EM), the E-step is often intractable without restrictive approximations to the posterior. We propose the use of GFlowNets, algorithms for sampling from an unnormalized density by learning a stochastic policy for sequential construction of samples, for this intractable E-step. By training GFlowNets to sample from the posterior over latents, we take advantage of their strengths as amortized variational inference algorithms for complex distributions over discrete structures. Our approach, GFlowNet-EM, enables the training of expressive LVMs with discrete compositional latents, as shown by experiments on non-context-free grammar induction and on images using discrete variational autoencoders (VAEs) without conditional independence enforced in the encoder.Comment: ICML 2023; code: https://github.com/GFNOrg/GFlowNet-E

PathLDM: Text conditioned Latent Diffusion Model for Histopathology

To achieve high-quality results, diffusion models must be trained on large datasets. This can be notably prohibitive for models in specialized domains, such as computational pathology. Conditioning on labeled data is known to help in data-efficient model training. Therefore, histopathology reports, which are rich in valuable clinical information, are an ideal choice as guidance for a histopathology generative model. In this paper, we introduce PathLDM, the first text-conditioned Latent Diffusion Model tailored for generating high-quality histopathology images. Leveraging the rich contextual information provided by pathology text reports, our approach fuses image and textual data to enhance the generation process. By utilizing GPT's capabilities to distill and summarize complex text reports, we establish an effective conditioning mechanism. Through strategic conditioning and necessary architectural enhancements, we achieved a SoTA FID score of 7.64 for text-to-image generation on the TCGA-BRCA dataset, significantly outperforming the closest text-conditioned competitor with FID 30.1

Learned representation-guided diffusion models for large-image generation

To synthesize high-fidelity samples, diffusion models typically require auxiliary data to guide the generation process. However, it is impractical to procure the painstaking patch-level annotation effort required in specialized domains like histopathology and satellite imagery; it is often performed by domain experts and involves hundreds of millions of patches. Modern-day self-supervised learning (SSL) representations encode rich semantic and visual information. In this paper, we posit that such representations are expressive enough to act as proxies to fine-grained human labels. We introduce a novel approach that trains diffusion models conditioned on embeddings from SSL. Our diffusion models successfully project these features back to high-quality histopathology and remote sensing images. In addition, we construct larger images by assembling spatially consistent patches inferred from SSL embeddings, preserving long-range dependencies. Augmenting real data by generating variations of real images improves downstream classifier accuracy for patch-level and larger, image-scale classification tasks. Our models are effective even on datasets not encountered during training, demonstrating their robustness and generalizability. Generating images from learned embeddings is agnostic to the source of the embeddings. The SSL embeddings used to generate a large image can either be extracted from a reference image, or sampled from an auxiliary model conditioned on any related modality (e.g. class labels, text, genomic data). As proof of concept, we introduce the text-to-large image synthesis paradigm where we successfully synthesize large pathology and satellite images out of text descriptions