Lipschitzness Effect of a Loss Function on Generalization Performance of Deep Neural Networks Trained by Adam and AdamW Optimizers

The generalization performance of deep neural networks with regard to the optimization algorithm is one of the major concerns in machine learning. This performance can be affected by various factors. In this paper, we theoretically prove that the Lipschitz constant of a loss function is an important factor to diminish the generalization error of the output model obtained by Adam or AdamW. The results can be used as a guideline for choosing the loss function when the optimization algorithm is Adam or AdamW. In addition, to evaluate the theoretical bound in a practical setting, we choose the human age estimation problem in computer vision. For assessing the generalization better, the training and test datasets are drawn from different distributions. Our experimental evaluation shows that the loss function with lower Lipschitz constant and maximum value improves the generalization of the model trained by Adam or AdamW.Comment: 13 pages, 6 figures, 3 table

Insulin Can Improve the Normal Function of the Brain by Preventing the Loss of the Neurons

Background: Insulin promotes the expression of genes related to brain function, thus preventing the neurodegeneration process. The present study was designed to find the neuroprotective effect of insulin by reducing neuron loss in the brain. Materials and Methods: In this study, 20 adult male NMRI mice were divided into two groups: control and insulin. The control group was intact, and the insulin group received 100 µL of insulin at a 72-hour interval by intraperitoneal (I.P.) injection for 30 days. At the end of the study, the brain was removed. The volume of the brain and the total number of neurons and glia were estimated by stereological techniques, and also the gene expression of NSR, PI3K, AKT, IGF-1, and FOXO-1 was measured using real-time PCR. Results: The results showed that the total number of neurons decreased in the control group compared to the experimental group. Furthermore, the expression of NSR, PI3K, AKT, IGF-1, and FOXO-1 genes was lower in the control group than in the insulin group. Conclusion: The results showed that treating mice with insulin prevented reducing the number of neurons and gene expression related to normal brain function. So, insulin could have neuroprotective effects against neuron loss. Insulin may be beneficial as a new approach to avoiding neuron loss in regenerative medicine

Characterizing the variation of small strain shear modulus for silt and sand during hydraulic hysteresis

Experimental studies have indicated that the small strain shear modulus, Gmax, of unsaturated silt and clay has a greater amount during imbibition than during drainage, when presented as a function of matric suction. However, due to material properties and inter-particle forces, different behavior is expected in the case of sand. Although considerable research has been devoted in recent years to characterize the behaviour of Gmax of sand during drainage, rather less attention has been paid to the effect of hydraulic hysteresis on Gmax and its variations during imbibition. In the study presented herein, an effort has been made to compare the Gmax behavior of specimens of silt and sand during hydraulic hysteresis. In this regard, a series of bender element tests were carried out in a modified triaxial test device with suction-saturation control to evaluate the impact of hydraulic hysteresis on Gmax for specimens of silt and sand. Trends between the Gmax and matric suction for unsaturated sand were found to be different from those for silty specimens. The variations in Gmax showed an up and down trend in both drainage and imbibition paths for sandy specimens, where plotted as a function of matric suction. Results also indicated smaller magnitudes of Gmax upon imbibition than those during drainage; a behavior which is believed to be attributed to variations in suction stress with matric suction. In silty specimens, a stiffer response was measured during imbibition which was hypothesized to be due to drainage-induced hardening experienced by the specimens that was not fully recovered during imbibition

Weighted minimum backward fréchet distance

The minimum backward Fréchet distance (MBFD) problem is a natural optimization problem for the weak Fréchet distance, a variant of the well-known Fréchet distance. In this problem, a threshold " and two polygonal curves, T1 and T2, are given. The objective is to find a pair of walks on T1 and T2, which minimizes the union of the portions of backward movements while the distance between the moving entities, at any time, is at most ". In this paper, we generalize this model to capture scenarios when the cost of backtracking on the input polygonal curves is not homogeneous. More specifically, each edge of T1 and T2 has an associated nonnegative weight. The cost of backtracking on an edge is the Euclidean length of backward movement on that edge multiplied by the corresponding weight. The objective is to find a pair of walks that minimizes the sum of the costs on the edges of the curves, while guaranteeing that the curves remain at weak Fréchet distance ϵ. We propose an exact algorithm whose run time and space complexity is O(n3), where n is the maximum number of the edges of T1 and T2

Weighted region problem in arrangement of lines

In this paper, a geometric shortest path problem in weighted regions is discussed. An arrangement of lines A, a source s, and a target t are given. The objective is to find a weighted shortest path, Πst, from s to t. Existing approximation algorithms for weighted shortest paths work within bounded regions (typically triangulated). To apply these algorithms to unbounded regions, such as arrangements of lines, there is a need to bound the regions. Here, we present a minimal region that contains Πst, called SP-Hull of A. It is a closed polygonal region that only depends on the geometry of the arrangement A and is independent of the weights. It is minimal in the sense that for any arrangement of lines A, it is possible to assign weights to the faces of A and choose s and t such that Πst is arbitrary close to the boundary of SP-Hull of A. We show that SP-Hull can be constructed in O(nlog n) time, where n is the number of lines in the arrangement. As a direct consequence we obtain a shortest path algorithm for weighted arrangements of lines

Weighted minimum backward Fréchet distance

The minimum backward Fréchet distance (MBFD) problem is a natural optimization problem for the weak Fréchet distance, a variant of the well-known Fréchet distance. In this problem, a threshold ε and two polygonal curves, T 1 and T 2 , are given. The objective is to find a pair of walks on T 1 and T 2 , which minimizes the union of the portions of backward movements (backtracking) while maintaining, at any time, a distance between the moving entities of at most ε. In this paper, we generalize this model to capture scenarios when the cost of backtracking on the input polygonal curves is not homogeneous. More specifically, each edge of T 1 and T 2 has an associated non-negative weight. The cost of backtracking on an edge is the Euclidean length of backward movement on that edge multiplied by the corresponding weight. The objective is to find a pair of walks that minimizes the sum of the costs on the edges of the curves, while guaranteeing that the weak traversal of the curves maintains a weak Fréchet distance of at most ε. We propose two exact algorithms, a simple algorithm with O(n 4 ) time and s

Minimum backward fréchet distance

We propose a new measure to capture similarity between polygonal curves, called the minimum backward Fréchet distance. It is a natural optimization on the weak Fréchet distance, a variant of the well-known Fréchet distance. More specifically, for a given threshold ε, we are searching for a pair of walks for two entities on the two input curves, T1 and T2, such that the union of the portions of backward movements is minimized and the distance between the two entities, at any time during the walk, is less than or equal to ". Our algorithm detects if no such pair of walks exists. This natural optimization problem appears in many applications in Geographical Information Systems, mobile networks and robotics. We provide an exact algorithm with time complexity of Ο (n2 log n) and space complexity of Ο (n2), where n is the maximum number of segments in the input polygonal curves