131 research outputs found
Likelihood based observability analysis and confidence intervals for predictions of dynamic models
Mechanistic dynamic models of biochemical networks such as Ordinary
Differential Equations (ODEs) contain unknown parameters like the reaction rate
constants and the initial concentrations of the compounds. The large number of
parameters as well as their nonlinear impact on the model responses hamper the
determination of confidence regions for parameter estimates. At the same time,
classical approaches translating the uncertainty of the parameters into
confidence intervals for model predictions are hardly feasible.
In this article it is shown that a so-called prediction profile likelihood
yields reliable confidence intervals for model predictions, despite arbitrarily
complex and high-dimensional shapes of the confidence regions for the estimated
parameters. Prediction confidence intervals of the dynamic states allow a
data-based observability analysis. The approach renders the issue of sampling a
high-dimensional parameter space into evaluating one-dimensional prediction
spaces. The method is also applicable if there are non-identifiable parameters
yielding to some insufficiently specified model predictions that can be
interpreted as non-observability. Moreover, a validation profile likelihood is
introduced that should be applied when noisy validation experiments are to be
interpreted.
The properties and applicability of the prediction and validation profile
likelihood approaches are demonstrated by two examples, a small and instructive
ODE model describing two consecutive reactions, and a realistic ODE model for
the MAP kinase signal transduction pathway. The presented general approach
constitutes a concept for observability analysis and for generating reliable
confidence intervals of model predictions, not only, but especially suitable
for mathematical models of biological systems
Less is More: Proxy Datasets in NAS approaches
Neural Architecture Search (NAS) defines the design of Neural Networks as a
search problem. Unfortunately, NAS is computationally intensive because of
various possibilities depending on the number of elements in the design and the
possible connections between them. In this work, we extensively analyze the
role of the dataset size based on several sampling approaches for reducing the
dataset size (unsupervised and supervised cases) as an agnostic approach to
reduce search time. We compared these techniques with four common NAS
approaches in NAS-Bench-201 in roughly 1,400 experiments on CIFAR-100. One of
our surprising findings is that in most cases we can reduce the amount of
training data to 25\%, consequently reducing search time to 25\%, while at the
same time maintaining the same accuracy as if training on the full dataset.
Additionally, some designs derived from subsets out-perform designs derived
from the full dataset by up to 22 p.p. accuracy
AudioCLIP: Extending CLIP to Image, Text and Audio
In the past, the rapidly evolving field of sound classification greatly
benefited from the application of methods from other domains. Today, we observe
the trend to fuse domain-specific tasks and approaches together, which provides
the community with new outstanding models.
In this work, we present an extension of the CLIP model that handles audio in
addition to text and images. Our proposed model incorporates the ESResNeXt
audio-model into the CLIP framework using the AudioSet dataset. Such a
combination enables the proposed model to perform bimodal and unimodal
classification and querying, while keeping CLIP's ability to generalize to
unseen datasets in a zero-shot inference fashion.
AudioCLIP achieves new state-of-the-art results in the Environmental Sound
Classification (ESC) task, out-performing other approaches by reaching
accuracies of 90.07% on the UrbanSound8K and 97.15% on the ESC-50 datasets.
Further it sets new baselines in the zero-shot ESC-task on the same datasets
68.78% and 69.40%, respectively).
Finally, we also assess the cross-modal querying performance of the proposed
model as well as the influence of full and partial training on the results. For
the sake of reproducibility, our code is published.Comment: submitted to GCPR 202
Waving Goodbye to Low-Res: A Diffusion-Wavelet Approach for Image Super-Resolution
This paper presents a novel Diffusion-Wavelet (DiWa) approach for
Single-Image Super-Resolution (SISR). It leverages the strengths of Denoising
Diffusion Probabilistic Models (DDPMs) and Discrete Wavelet Transformation
(DWT). By enabling DDPMs to operate in the DWT domain, our DDPM models
effectively hallucinate high-frequency information for super-resolved images on
the wavelet spectrum, resulting in high-quality and detailed reconstructions in
image space. Quantitatively, we outperform state-of-the-art diffusion-based
SISR methods, namely SR3 and SRDiff, regarding PSNR, SSIM, and LPIPS on both
face (8x scaling) and general (4x scaling) SR benchmarks. Meanwhile, using DWT
enabled us to use fewer parameters than the compared models: 92M parameters
instead of 550M compared to SR3 and 9.3M instead of 12M compared to SRDiff.
Additionally, our method outperforms other state-of-the-art generative methods
on classical general SR datasets while saving inference time. Finally, our work
highlights its potential for various applications
YODA: You Only Diffuse Areas. An Area-Masked Diffusion Approach For Image Super-Resolution
This work introduces "You Only Diffuse Areas" (YODA), a novel method for
partial diffusion in Single-Image Super-Resolution (SISR). The core idea is to
utilize diffusion selectively on spatial regions based on attention maps
derived from the low-resolution image and the current time step in the
diffusion process. This time-dependent targeting enables a more effective
conversion to high-resolution outputs by focusing on areas that benefit the
most from the iterative refinement process, i.e., detail-rich objects. We
empirically validate YODA by extending leading diffusion-based SISR methods SR3
and SRDiff. Our experiments demonstrate new state-of-the-art performance gains
in face and general SR across PSNR, SSIM, and LPIPS metrics. A notable finding
is YODA's stabilization effect on training by reducing color shifts, especially
when induced by small batch sizes, potentially contributing to
resource-constrained scenarios. The proposed spatial and temporal adaptive
diffusion mechanism opens promising research directions, including developing
enhanced attention map extraction techniques and optimizing inference latency
based on sparser diffusion.Comment: Brian B. Moser and Stanislav Frolov contributed equall
- …