Deep Information Networks

We describe a novel classifier with a tree structure, designed using information theory concepts. This Information Network is made of information nodes, that compress the input data, and multiplexers, that connect two or more input nodes to an output node. Each information node is trained, independently of the others, to minimize a local cost function that minimizes the mutual information between its input and output with the constraint of keeping a given mutual information between its output and the target (information bottleneck). We show that the system is able to provide good results in terms of accuracy, while it shows many advantages in terms of modularity and reduced complexity

Contributions to Efficient Machine Learning

L'abstract è presente nell'allegato / the abstract is in the attachmen

Semi-Supervised GNSS Scintillations Detection Based on DeepInfomax

This work focuses on a machine learning based detection of iono-spheric scintillation events affecting Global Navigation Satellite System (GNSS) signals. We here extend the recent detection results based on Decision Trees, designing a semi-supervised detection system based on the DeepInfomax approach recently proposed. The paper shows that it is possible to achieve good classification accuracy while reducing the amount of time that human experts must spend manually labelling the datasets for the training of supervised algorithms. The proposed method is scalable and reduces the required percentage of annotated samples to achieve a given performance, making it a viable candidate for a realistic deployment of scintillation detection in software defined GNSS receivers

MINDE: Mutual Information Neural Diffusion Estimation

In this work we present a new method for the estimation of Mutual Information (MI) between random variables. Our approach is based on an original interpretation of the Girsanov theorem, which allows us to use score-based diffusion models to estimate the Kullback Leibler divergence between two densities as a difference between their score functions. As a by-product, our method also enables the estimation of the entropy of random variables. Armed with such building blocks, we present a general recipe to measure MI, which unfolds in two directions: one uses conditional diffusion process, whereas the other uses joint diffusion processes that allow simultaneous modelling of two random variables. Our results, which derive from a thorough experimental protocol over all the variants of our approach, indicate that our method is more accurate than the main alternatives from the literature, especially for challenging distributions. Furthermore, our methods pass MI self-consistency tests, including data processing and additivity under independence, which instead are a pain-point of existing methods

On the notion of value. A comparative analysis between economic and biophysical approaches

The plurality of dimensions and topics covered by the SDGs reflects the need to assess the value of organizations, cities, and societies using a holistic approach that considers different dimensions and criteria. It is much needed to shift towards inter-disciplinary, multi-criteria and integrated perspectives, opening the door to views able to consider different scientific points of view when assessing the most “valuable” pillars in human societies. This need highlights a controversial question: “what do we mean when we refer to a concept so broad such as the one of “value” and its measurement”? The concept of value and welfare have changed throughout the years, also in relation to the historical context and societal structure and needs of the time. But time has not been the only factor in differentiating value theories. While most organically structured definitions of value have originated, as expected, from the developments of the economic discipline, this issue has also been addressed by scientists belonging to the biophysical realm. In this paper, a comparative overview of the main economic and biophysical value theories, developing from very different epistemological backgrounds, is provided. Results suggest the need to foster inter-disciplinary communication on the notion of value, which is an abstract construct at the root of our societies and economies

Continuous-Time Functional Diffusion Processes

We introduce Functional Diffusion Processes (FDPs), which generalize score-based diffusion models to infinite-dimensional function spaces. FDPs require a new mathematical framework to describe the forward and backward dynamics, and several extensions to derive practical training objectives. These include infinite-dimensional versions of Girsanov theorem, in order to be able to compute an ELBO, and of the sampling theorem, in order to guarantee that functional evaluations in a countable set of points are equivalent to infinite-dimensional functions. We use FDPs to build a new breed of generative models in function spaces, which do not require specialized network architectures, and that can work with any kind of continuous data. Our results on real data show that FDPs achieve high-quality image generation, using a simple MLP architecture with orders of magnitude fewer parameters than existing diffusion models.Comment: Under revie

One-Line-of-Code Data Mollification Improves Optimization of Likelihood-based Generative Models

Generative Models (GMs) have attracted considerable attention due to their tremendous success in various domains, such as computer vision where they are capable to generate impressive realistic-looking images. Likelihood-based GMs are attractive due to the possibility to generate new data by a single model evaluation. However, they typically achieve lower sample quality compared to state-of-the-art score-based diffusion models (DMs). This paper provides a significant step in the direction of addressing this limitation. The idea is to borrow one of the strengths of score-based DMs, which is the ability to perform accurate density estimation in low-density regions and to address manifold overfitting by means of data mollification. We connect data mollification through the addition of Gaussian noise to Gaussian homotopy, which is a well-known technique to improve optimization. Data mollification can be implemented by adding one line of code in the optimization loop, and we demonstrate that this provides a boost in generation quality of likelihood-based GMs, without computational overheads. We report results on image data sets with popular likelihood-based GMs, including variants of variational autoencoders and normalizing flows, showing large improvements in FID score

Small-Angle X-ray Scattering Unveils the Internal Structure of Lipid Nanoparticles

Lipid nanoparticles own a remarkable potential in nanomedicine, only partially disclosed. While the clinical use of liposomes and cationic lipid-nucleic acid complexes is well-established, liquid lipid nanoparticles (nanoemulsions), solid lipid nanoparticles, and nanostructured lipid carriers have even greater potential. However, they face obstacles in being used in clinics due to a lack of understanding about the molecular mechanisms controlling their drug loading and release, interactions with the biological environment (such as the protein corona), and shelf-life stability. To create effective drug delivery carriers and successfully translate bench research to clinical settings, it is crucial to have a thorough understanding of the internal structure of lipid nanoparticles. Through synchrotron small-angle X-ray scattering experiments, we determined the spatial distribution and internal structure of the nanoparticles' lipid, surfactant, and the water in them. The nanoparticles themselves have a barrel-like shape that consists of coplanar lipid platelets (specifically cetyl palmitate) that are partially covered by polysorbate 80 surfactant and retain a small amount of hydration water. Although the platelet structure was expected, the presence of surfactant molecules forming sticky patches between adjacent platelets challenges the classical core-shell model used to describe solid lipid nanoparticles. Additionally, the surfactant partially covers the water-nanoparticle interface, allowing certain lipid regions to come into direct contact with surrounding water. These structural features play a significant role in drug loading and release, biological fluid interaction, and nanoparticle stability, making these findings valuable for the rational design of lipid-based nanoparticles.Comment: 22 pages, 11 figure