8 research outputs found
Improved DDIM Sampling with Moment Matching Gaussian Mixtures
We propose using a Gaussian Mixture Model (GMM) as reverse transition
operator (kernel) within the Denoising Diffusion Implicit Models (DDIM)
framework, which is one of the most widely used approaches for accelerated
sampling from pre-trained Denoising Diffusion Probabilistic Models (DDPM).
Specifically we match the first and second order central moments of the DDPM
forward marginals by constraining the parameters of the GMM. We see that moment
matching is sufficient to obtain samples with equal or better quality than the
original DDIM with Gaussian kernels. We provide experimental results with
unconditional models trained on CelebAHQ and FFHQ and class-conditional models
trained on ImageNet datasets respectively. Our results suggest that using the
GMM kernel leads to significant improvements in the quality of the generated
samples when the number of sampling steps is small, as measured by FID and IS
metrics. For example on ImageNet 256x256, using 10 sampling steps, we achieve a
FID of 6.94 and IS of 207.85 with a GMM kernel compared to 10.15 and 196.73
respectively with a Gaussian kernel.Comment: 29 pages, 14 figures; Analysis of DDIM-GMM as a multimodal denoiser;
Additional experiments on LSUN datasets and text-to-image generation with
Stable Diffusion; Comparison with DPM-Solver; Ablations on GMM parameters;
Updated equations with bold font for vectors and matrice
Spiking Patterns and Their Functional Implications in the Antennal Lobe of the Tobacco Hornworm \u3cem\u3eManduca sexta\u3c/em\u3e
Bursting as well as tonic firing patterns have been described in various sensory systems. In the olfactory system, spontaneous bursts have been observed in neurons distributed across several synaptic levels, from the periphery, to the olfactory bulb (OB) and to the olfactory cortex. Several in vitro studies indicate that spontaneous firing patterns may be viewed as “fingerprints” of different types of neurons that exhibit distinct functions in the OB. It is still not known, however, if and how neuronal burstiness is correlated with the coding of natural olfactory stimuli. We thus conducted an in vivo study to probe this question in the OB equivalent structure of insects, the antennal lobe (AL) of the tobacco hornworm Manduca sexta. We found that in the moth\u27s AL, both projection (output) neurons (PNs) and local interneurons (LNs) are spontaneously active, but PNs tend to produce spike bursts while LNs fire more regularly. In addition, we found that the burstiness of PNs is correlated with the strength of their responses to odor stimulation – the more bursting the stronger their responses to odors. Moreover, the burstiness of PNs was also positively correlated with the spontaneous firing rate of these neurons, and pharmacological reduction of bursting resulted in a decrease of the neurons\u27 responsiveness. These results suggest that neuronal burstiness reflects a physiological state of these neurons that is directly linked to their response characteristics
Spiking Patterns and Their Functional Implications in the Antennal Lobe of the Tobacco Hornworm Manduca sexta
Bursting as well as tonic firing patterns have been described in various sensory systems. In the olfactory system, spontaneous bursts have been observed in neurons distributed across several synaptic levels, from the periphery, to the olfactory bulb (OB) and to the olfactory cortex. Several in vitro studies indicate that spontaneous firing patterns may be viewed as “fingerprints” of different types of neurons that exhibit distinct functions in the OB. It is still not known, however, if and how neuronal burstiness is correlated with the coding of natural olfactory stimuli. We thus conducted an in vivo study to probe this question in the OB equivalent structure of insects, the antennal lobe (AL) of the tobacco hornworm Manduca sexta. We found that in the moth's AL, both projection (output) neurons (PNs) and local interneurons (LNs) are spontaneously active, but PNs tend to produce spike bursts while LNs fire more regularly. In addition, we found that the burstiness of PNs is correlated with the strength of their responses to odor stimulation – the more bursting the stronger their responses to odors. Moreover, the burstiness of PNs was also positively correlated with the spontaneous firing rate of these neurons, and pharmacological reduction of bursting resulted in a decrease of the neurons' responsiveness. These results suggest that neuronal burstiness reflects a physiological state of these neurons that is directly linked to their response characteristics
Recommended from our members
Machine Learning Methods for Microarray Data Analysis
Microarrays emerged in the 1990s as a consequence of the efforts to speed up the process of drug discovery. They revolutionized molecular biological research by enabling monitoring of thousands of genes together. Typical microarray experiments measure the expression levels of a large numberof genes on very few tissue samples. The resulting sparsity of data presents major challenges to statistical methods used to perform any kind of analysis on this data. This research posits that phenotypic classification and prediction serve as good objective functions for both optimization and evaluation of microarray data analysis methods. This is because classification measures whatis needed for diagnostics and provides quantitative performance measures such as leave-one-out (LOO) or held-out prediction accuracy and confidence. Under the classification framework, various microarray data normalization procedures are evaluated using a class label hypothesis testing framework and also employing Support Vector Machines (SVM) and linear discriminant based classifiers. A novel normalization technique based on minimizing the squared correlation coefficients between expression levels of gene pairs is proposed and evaluated along with the other methods. Our results suggest that most normalization methods helped classification on the datasets considered except the rank method, most likely due to its quantization effects.Another contribution of this research is in developing machine learning methods for incorporating an independent source of information, in the form of gene annotations, to analyze microarray data. Recently, genes of many organisms have been annotated with terms from a limited vocabulary called Gene Ontologies (GO), describing the genes' roles in various biological processes, molecular functions and their locations within the cell. Novel probabilistic generative models are proposed for clustering genes using both their expression levels and GO tags. These models are similar in essence to the ones used for multimodal data, such as images and words, with learning and inference done in a Bayesian framework. The multimodal generative models are used for phenotypic class prediction. More specifically, the problems of phenotype prediction for static gene expression data and state prediction for time-course data are emphasized. Using GO tags for organisms whose genes have been studied more comprehensively leads to an improvement in prediction. Our methods also have the potential to provide a way to assess the quality of available GO tags for the genes of various model organisms
Color and color constancy in a translation model for object recognition
Color is of interest to those working in computer vision largely because it is assumed to be helpful for recognition. This assumption has driven much work in color based image indexing, and computational color constancy. However, in many ways, indexing is a poor model for recognition. In this paper we use a recently developed statistical model of recognition which learns to link image region features with words, based on a large unstructured data set. The system is general in that it learns what is recognizable given the data. It also supports a principled testing paradigm which we exploit here to evaluate the use of color. In particular, we look at color space choice, degradation due to illumination change, and dealing with this degradation. We evaluate two general approaches to dealing with this color constancy problem. Specifically we address whether it is better to build color variation due to illumination into a recognition system, or, instead, apply color constancy preprocessing to images before they are processed by the recognition system