Investigation into the Training Dynamics of Learned Optimizers

Optimization is an integral part of modern deep learning. Recently, the concept of learned optimizers has emerged as a way to accelerate this optimization process by replacing traditional, hand-crafted algorithms with meta-learned functions. Despite the initial promising results of these methods, issues with stability and generalization still remain, limiting their practical use. Moreover, their inner workings and behavior under different conditions are not yet fully understood, making it difficult to come up with improvements. For this reason, our work examines their optimization trajectories from the perspective of network architecture symmetries and parameter update distributions. Furthermore, by contrasting the learned optimizers with their manually designed counterparts, we identify several key insights that demonstrate how each approach can benefit from the strengths of the other

Built-up structure criticality

The built-up land represents an important type of overall landscape. In this paper the built-up land structure in the largest cities in the Czech Republic and selected cities in the U.S.A. is analysed using the framework of statistical physics. We calculate the variance of the built-up area and the number variance of built-up landed plots in discs. In both cases the variance as a function of a disc radius follows a power law. The obtained values of power law exponents are comparable through different cities. The study is based on cadastral data from the Czech Republic and building footprints from GIS data in the U.S.A.Comment: 14 pages, 11 figure

The distribution of landed property

The distribution of property is established through various mechanisms. In this paper we study the acreage distribution of land plots owned by natural persons in the Zl\'{\i}n Region of the Czech Republic. We show that the data are explained in terms of a simple model in which the inheritance and market behavior are combined.Comment: The distribution of landed propert

Chaos in a one-dimensional integrable quantum system

We study a simple one-dimensional quantum system on a circle with n scale free point interactions. The spectrum of this system is discrete and expressible as a solution of an explicit secular equation. However, its statistical properties are nontrivial. The level spacing distribution between its neighboring odd and even levels displays a surprising agreement with the prediction obtained for the Gaussian Orthogonal Ensemble of random matrices.Comment: 11 pages, 5 figure

Meningioma microstructure assessed by diffusion MRI : An investigation of the source of mean diffusivity and fractional anisotropy by quantitative histology

BACKGROUND: Mean diffusivity (MD) and fractional anisotropy (FA) from diffusion MRI (dMRI) have been associated with cell density and tissue anisotropy across tumors, but it is unknown whether these associations persist at the microscopic level.PURPOSE: To quantify the degree to which cell density and anisotropy, as determined from histology, account for the intra-tumor variability of MD and FA in meningioma tumors. Furthermore, to clarify whether other histological features account for additional intra-tumor variability of dMRI parameters.MATERIALS AND METHODS: We performed ex-vivo dMRI at 200 μm isotropic resolution and histological imaging of 16 excised meningioma tumor samples. Diffusion tensor imaging (DTI) was used to map MD and FA, as well as the in-plane FA (FA IP). Histology images were analyzed in terms of cell nuclei density (CD) and structure anisotropy (SA; obtained from structure tensor analysis) and were used separately in a regression analysis to predict MD and FA IP, respectively. A convolutional neural network (CNN) was also trained to predict the dMRI parameters from histology patches. The association between MRI and histology was analyzed in terms of out-of-sample (R 2 OS) on the intra-tumor level and within-sample R 2 across tumors. Regions where the dMRI parameters were poorly predicted from histology were analyzed to identify features apart from CD and SA that could influence MD and FA IP, respectively. RESULTS: Cell density assessed by histology poorly explained intra-tumor variability of MD at the mesoscopic level (200 μm), as median R 2 OS = 0.04 (interquartile range 0.01-0.26). Structure anisotropy explained more of the variation in FA IP (median R 2 OS = 0.31, 0.20-0.42). Samples with low R 2 OS for FA IP exhibited low variations throughout the samples and thus low explainable variability, however, this was not the case for MD. Across tumors, CD and SA were clearly associated with MD (R 2 = 0.60) and FA IP (R 2 = 0.81), respectively. In 37% of the samples (6 out of 16), cell density did not explain intra-tumor variability of MD when compared to the degree explained by the CNN. Tumor vascularization, psammoma bodies, microcysts, and tissue cohesivity were associated with bias in MD prediction based solely on CD. Our results support that FA IP is high in the presence of elongated and aligned cell structures, but low otherwise. CONCLUSION: Cell density and structure anisotropy account for variability in MD and FA IP across tumors but cell density does not explain MD variations within the tumor, which means that low or high values of MD locally may not always reflect high or low tumor cell density. Features beyond cell density need to be considered when interpreting MD