Intrinsic dimension estimation for locally undersampled data

Identifying the minimal number of parameters needed to describe a dataset is a challenging problem known in the literature as intrinsic dimension estimation. All the existing intrinsic dimension estimators are not reliable whenever the dataset is locally undersampled, and this is at the core of the so called curse of dimensionality. Here we introduce a new intrinsic dimension estimator that leverages on simple properties of the tangent space of a manifold and extends the usual correlation integral estimator to alleviate the extreme undersampling problem. Based on this insight, we explore a multiscale generalization of the algorithm that is capable of (i) identifying multiple dimensionalities in a dataset, and (ii) providing accurate estimates of the intrinsic dimension of extremely curved manifolds. We test the method on manifolds generated from global transformations of high-contrast images, relevant for invariant object recognition and considered a challenge for state-of-the-art intrinsic dimension estimators

Counting the learnable functions of geometrically structured data

Cover's function counting theorem is a milestone in the theory of artificial neural networks. It provides an answer to the fundamental question of determining how many binary assignments (dichotomies) of p points in n dimensions can be linearly realized. Regrettably, it has proved hard to extend the same approach to more advanced problems than the classification of points. In particular, an emerging necessity is to find methods to deal with geometrically structured data, and specifically with non-point-like patterns. A prominent case is that of invariant recognition, whereby identification of a stimulus is insensitive to irrelevant transformations on the inputs (such as rotations or changes in perspective in an image). An object is thus represented by an extended perceptual manifold, consisting of inputs that are classified similarly. Here, we develop a function counting theory for structured data of this kind, by extending Cover's combinatorial technique, and we derive analytical expressions for the average number of dichotomies of generically correlated sets of patterns. As an application, we obtain a closed formula for the capacity of a binary classifier trained to distinguish general polytopes of any dimension. These results extend our theoretical understanding of the role of data structure in machine learning, and provide useful quantitative tools for the analysis of generalization, feature extraction, and invariant object recognition by neural networks

Large deviations of the free energy in the p-spin glass spherical model

We investigate the behavior of the rare fluctuations of the free energy in the p-spin spherical model, evaluating the corresponding rate function via the G\ue4rtner-Ellis theorem. This approach requires the knowledge of the analytic continuation of the disorder-averaged replicated partition function to arbitrary real number of replicas. In zero external magnetic field, we show via a one-step replica symmetry breaking calculation that the rate function is infinite for fluctuations of the free energy above its typical value, corresponding to an anomalous, superextensive suppression of rare fluctuations. We extend this calculation to nonzero magnetic field, showing that in this case this very large deviation disappears and we try to motivate this finding in light of a geometrical interpretation of the scaled cumulant generating function

EMERGENT COLLECTIVE PHENOMENA IN QUANTUM MANY-BODY SYSTEMS

Motivated by a recent proposal by Lev and coworkers, in the first part of this thesis, we will perform a theoretical investigation of a new class of quantum simulators: the so called multimode disordered Dicke simulators. Our approach is mostly inspired from Statistical Mechanics: indeed we will merge together exact results obtained in the context of the Dicke model by Hepp and Lieb, with known results on disordered systems and neural networks. In this way we will be able to generalize the standard approach to the superradiant phase transition to the disordered case. As a byproduct of this analysis we will argue that this new class of quantum simulators (properly engineered) may be an alternative (or complementary) route toward quantum computation. Also the second part of the thesis has a \u201dquantum simulators motivation\u201d. Recently Bloch\u2019s group implemented an Ising quantum magnet with long-range antiferro- magnetic interactions, which exhibits a peculiar devil\u2019s staircase phase diagram, predicted long ago by Bak and Bruinsma. This result, joined with recent theoretical investigations by Lesanowsky and coworkers suggested to us to reconsider these spin models in the context of the fractional quantum Hall effect (FQHE). In the second part of this thesis we will show that the quantum Hall Hamitonian projected on the lowest Landau level can be mapped, in the so called thin torus limit, on the lattice gas studied by Bak and Bruinsma. This observation will lead us to predict a devil\u2019s staircase scenario for the Hall conductance as a function of the magnetic field. This work stimulated us to investigate the connection between Laughlin wave function and Tao-Thouless states, that we will explore in the last section of the second part

Universal mean-field upper bound for the generalization gap of deep neural networks

Modern deep neural networks (DNNs) represent a formidable challenge for theorists: according to the commonly accepted probabilistic framework that describes their performance, these architectures should overfit due to the huge number of parameters to train, but in practice they do not. Here we employ results from replica mean field theory to compute the generalization gap of machine learning models with quenched features, in the teacher-student scenario and for regression problems with quadratic loss function. Notably, this framework includes the case of DNNs where the last layer is optimized given a specific realization of the remaining weights. We show how these results-combined with ideas from statistical learning theory-provide a stringent asymptotic upper bound on the generalization gap of fully trained DNN as a function of the size of the dataset P. In particular, in the limit of large P and N-out (where N-out is the size of the last layer) and N-out << P, the generalization gap approaches zero faster than 2N(out)/P, for any choice of both architecture and teacher function. Notably, this result greatly improves existing bounds from statistical learning theory. We test our predictions on a broad range of architectures, from toy fully connected neural networks with few hidden layers to state-of-the-art deep convolutional neural networks

Local Kernel Renormalization as a mechanism for feature learning in overparametrized Convolutional Neural Networks

Feature learning, or the ability of deep neural networks to automatically learn relevant features from raw data, underlies their exceptional capability to solve complex tasks. However, feature learning seems to be realized in different ways in fully-connected (FC) or convolutional architectures (CNNs). Empirical evidence shows that FC neural networks in the infinite-width limit eventually outperform their finite-width counterparts. Since the kernel that describes infinite-width networks does not evolve during training, whatever form of feature learning occurs in deep FC architectures is not very helpful in improving generalization. On the other hand, state-of-the-art architectures with convolutional layers achieve optimal performances in the finite-width regime, suggesting that an effective form of feature learning emerges in this case. In this work, we present a simple theoretical framework that provides a rationale for these differences, in one hidden layer networks. First, we show that the generalization performance of a finite-width FC network can be obtained by an infinite-width network, with a suitable choice of the Gaussian priors. Second, we derive a finite-width effective action for an architecture with one convolutional hidden layer and compare it with the result available for FC networks. Remarkably, we identify a completely different form of kernel renormalization: whereas the kernel of the FC architecture is just globally renormalized by a single scalar parameter, the CNN kernel undergoes a local renormalization, meaning that the network can select the local components that will contribute to the final prediction in a data-dependent way. This finding highlights a simple mechanism for feature learning that can take place in overparametrized shallow CNNs, but not in shallow FC architectures or in locally connected neural networks without weight sharing.Comment: 22 pages, 5 figures, 2 tables. Comments are welcom

Bone Involvement in Systemic Lupus Erythematosus

Systemic lupus erythematosus (SLE) is a chronic autoimmune disease characterized by a wide variability of clinical manifestations due to the potential involvement of several tissues and internal organs, with a relapsing and remitting course. Dysregulation of innate and adaptive immune systems, due to genetic, hormonal and environmental factors, may be responsible for a broad spectrum of clinical manifestations, affecting quality of life, morbidity and mortality. Bone involvement represents one of the most common cause of morbidity and disability in SLE. Particularly, an increased incidence of osteoporosis, avascular necrosis of bone and osteomyelitis has been observed in SLE patients compared to the general population. Moreover, due to the improvement in diagnosis and therapy, the survival of SLE patient has improved, increasing long-term morbidities, including osteoporosis and related fractures. This review aims to highlight bone manifestations in SLE patients, deepening underlying etiopathogenetic mechanisms, diagnostic tools and available treatment

The optical links for the trigger upgrade of the Drift Tube in CMS

The first phase of the upgrade of the electronics of Drift Tubes (DT) in the CMS experiment is reported. It consists of the translation of the readout and trigger data from electrical into optical and their transmission from the CMS experimental cavern to the counting room. Collecting the full information of the DT chambers in the counting room allows the development of new trigger hardware and algorithms

Anti-cyclic-citrullinated-protein-antibodies in psoriatic arthritis patients: how autoimmune dysregulation could affect clinical characteristics, retention rate of methotrexate monotherapy and first line biotechnological drug survival. A single center retrospective study

Aim: Occasional findings of anti-cyclic-citrullinated-protein-antibodies (anti-CCP) were rarely observed in psoriatic arthritis (PsA). The aim of our study is to evaluate whether the presence of anti-CCP can determine different clinical subsets and influence methotrexate monotherapy survival, and biotechnological drug retention rate. Methods: We conducted a retrospective study on PsA patients. All patients were required to fulfill the CASPAR criteria for PsA, and to present juxta-articular osteo-proliferative signs at X-ray. The exclusion criteria were age less than 18 years old, satisfaction of rheumatoid arthritis classification criteria, and seropositivity for rheumatoid factor. Clinical characteristics, anti-CCP titer, drug survival and comorbidities information were recorded for each patient. Statistical significance was set at p ⩽ 0.05. Results: Of 407 patients with PsA screened 113 were recruited. Twelve patients were anti-CCP positive. Methotrexate monotherapy survival was shorter in patients with anti-CCP (150 ± 48.3 weeks versus 535.3 ± 65.3 weeks; p = 0.026) [discontinuation risk hazard ratio (HR) = 2.389, 95% confidence interval (CI) 1.043, 5.473; p = 0.039] than those without. Significant shorter survival of first-line biotechnological drugs (b-DMARDs) was observed in the anti-CCP positive group than in that without (102.05 ± 24.4 weeks versus 271.6 ± 41.7 weeks; p = 0.005) with higher discontinuation risk (HR = 3.230, 95% CI 1.299, 8.028; p = 0.012). A significant higher rate of multi-failure (more than second-line b-DMARDs) was found in anti-CCP positive patients than in those without (50% versus 14%, p = 0.035). Conclusion: Anti-CCP in PsA could be suggestive of more severe disease, with worse drug survival of both methotrexate monotherapy and first-line b-DMARDs, and higher chance to be b-DMARDs multi-failure. So, they can be considered for more intensive clinical management of these patients