Avoiding overfitting of multilayer perceptrons by training derivatives

Resistance to overfitting is observed for neural networks trained with extended backpropagation algorithm. In addition to target values, its cost function uses derivatives of those up to the $4^{\mathrm{th}}$ order. For common applications of neural networks, high order derivatives are not readily available, so simpler cases are considered: training network to approximate analytical function inside 2D and 5D domains and solving Poisson equation inside a 2D circle. For function approximation, the cost is a sum of squared differences between output and target as well as their derivatives with respect to the input. Differential equations are usually solved by putting a multilayer perceptron in place of unknown function and training its weights, so that equation holds within some margin of error. Commonly used cost is the equation's residual squared. Added terms are squared derivatives of said residual with respect to the independent variables. To investigate overfitting, the cost is minimized for points of regular grids with various spacing, and its root mean is compared with its value on much denser test set. Fully connected perceptrons with six hidden layers and $2\cdot10^{4}$, $1\cdot10^{6}$ and $5\cdot10^{6}$ weights in total are trained with Rprop until cost changes by less than 10% for last 1000 epochs, or when the $10000^{\mathrm{th}}$ epoch is reached. Training the network with $5\cdot10^{6}$ weights to represent simple 2D function using 10 points with 8 extra derivatives in each produces cost test to train ratio of $1.5$, whereas for classical backpropagation in comparable conditions this ratio is $2\cdot10^{4}$

Variable order Mittag-Leffler fractional operators on isolated time scales and application to the calculus of variations

We introduce new fractional operators of variable order on isolated time scales with Mittag-Leffler kernels. This allows a general formulation of a class of fractional variational problems involving variable-order difference operators. Main results give fractional integration by parts formulas and necessary optimality conditions of Euler-Lagrange type.Comment: This is a preprint of a paper whose final and definite form is with Springe

Association of sleep, screen time and physical activity with overweight and obesity in Mexico

PURPOSE: Approximately 70% of adults in Mexico are overweight or obese. Unhealthy lifestyle behaviors are also prevalent. We examined the association of three lifestyle behaviors with body mass index (BMI) categories in adults from Mexico. METHODS: We used publicly available data from the ENSANUT 2016 survey (n = 6419). BMI was used to categorize participants. Differences in sleep duration, suffering from symptoms of insomnia, TV watching time, time in front of any screen, vigorous physical activity (yes vs no), moderate physical activity (> 30 min/day-yes vs. no) and walking (> 60 min/day-yes vs. no) were compared across BMI groups using adjusted linear and logistic regression analyses. RESULTS: Thirty-nine percent of participants were overweight and 37% obese. Time in front of TV, in front of any screen, sleep duration and physical activity were significantly associated with overweight and obesity. Compared to normal weight participants, participants in the obese II category spend on average 0.60 h/day (95% CI 0.36-0.84, p = 0.001) and participants in the obese III category 0.54 h/day (95% CI 0.19-0.89, p < 0.001) more in front of any screen; participants in the obese II category reported 0.55 h/day less sleep (95% CI - 0.67 to - 0.43, p < 0.001); participants in the obese III category were less likely to engage in vigorous activity (OR = 0.60, 95% CI 0.43-0.84, p ≤ 0.003), or walking (OR = 0.65, 95% CI 0.49-0.88, p = 0.005). CONCLUSION: Screen time, sleeping hours, and physical activity were associated with overweight and obesity. However, these associations were not consistent across all BMI categories. Assuming established causal connections, overweight individuals and individuals with obesity would benefit from reduced screen time and engaging in moderate/vigorous physical activity. LEVEL OF EVIDENCE: Level III: observational case-control analytic study