A Three-regime Model of Network Pruning

Recent work has highlighted the complex influence training hyperparameters, e.g., the number of training epochs, can have on the prunability of machine learning models. Perhaps surprisingly, a systematic approach to predict precisely how adjusting a specific hyperparameter will affect prunability remains elusive. To address this gap, we introduce a phenomenological model grounded in the statistical mechanics of learning. Our approach uses temperature-like and load-like parameters to model the impact of neural network (NN) training hyperparameters on pruning performance. A key empirical result we identify is a sharp transition phenomenon: depending on the value of a load-like parameter in the pruned model, increasing the value of a temperature-like parameter in the pre-pruned model may either enhance or impair subsequent pruning performance. Based on this transition, we build a three-regime model by taxonomizing the global structure of the pruned NN loss landscape. Our model reveals that the dichotomous effect of high temperature is associated with transitions between distinct types of global structures in the post-pruned model. Based on our results, we present three case-studies: 1) determining whether to increase or decrease a hyperparameter for improved pruning; 2) selecting the best model to prune from a family of models; and 3) tuning the hyperparameter of the Sharpness Aware Minimization method for better pruning performance.Comment: ICML 202

A bargaining procedure leading to the serial rule in games with veto players

This paper studies an allocation procedure for coalitional games with veto players. The procedure is similar to the one presented by Arin and Feltkamp (J Math Econ 43:855-870, 2007), which is based on Dagan et al. (Games Econ Behav 18:55-72, 1997). A distinguished player makes a proposal that the remaining players must accept or reject, and conflict is solved bilaterally between the rejector and the proposer. We allow the proposer to make sequential proposals over several periods. If responders are myopic maximizers (i.e. consider each period in isolation), the only equilibrium outcome is the serial rule of Arin and Feltkamp (Eur J Oper Res 216:208-213, 2012) regardless of the order of moves. If all players are fully rational, the serial rule still arises as the unique subgame perfect equilibrium outcome if the order of moves is such that stronger players respond to the proposal after weaker ones

Fiscal shocks and real exchange rate dynamics: Some evidence for Latin America

This paper analyses the effects of fiscal shocks using a two-country macroeconomic model for output, labour input, government spending and relative prices which provides the orthogonality restrictions for obtaining the structural shocks. Dynamic simulation techniques are then applied, in particular to shed light on the possible effects of fiscal imbalances on the real exchange rate in the case of six Latin American countries. Using quarterly data over the period 1980-2006, we find that in a majority of cases fiscal shocks are the main driving force of real exchange rate fluctuations

Molecular evidence of adenosine deaminase linking adenosine A2A receptor and CD26 proteins

Adenosine is an endogenous purine nucleoside that acts in all living systems as a homeostatic network regulator through many pathways, which are adenosine receptor (AR)-dependent and -independent. From a metabolic point of view, adenosine deaminase (ADA) is an essential protein in the regulation of the total intracellular and extracellular adenosine in a tissue. In addition to its cytosolic localization, ADA is also expressed as an ecto-enzyme on the surface of different cells. Dipeptidyl peptidase IV (CD26) and some ARs act as binding proteins for extracellular ADA in humans. Since CD26 and ARs interact with ADA at opposite sites, we have investigated if ADA can function as a cell-to-cell communication molecule by bridging the anchoring molecules CD26 and A2AR present on the surfaces of the interacting cells. By combining site-directed mutagenesis of ADA amino acids involved in binding to A2AR and a modification of the bioluminescence resonance energy transfer (BRET) technique that allows detection of interactions between two proteins expressed in different cell populations with low steric hindrance (NanoBRET), we show direct evidence of the specific formation of trimeric complexes CD26-ADA-A2AR involving two cells. By dynamic mass redistribution assays and ligand binding experiments, we also demonstrate that A2AR-NanoLuc fusion proteins are functional. The existence of this ternary complex is in good agreement with the hypothesis that ADA could bridge T-cells (expressing CD26) and dendritic cells (expressing A2AR). This is a new metabolic function for ecto-ADA that, being a single chain protein, it has been considered as an example of moonlighting protein, because it performs more than one functional role (as a catalyst, a costimulator, an allosteric modulator and a cell-to-cell connector) without partitioning these functions in different subunits

Meta-analysis of nature conservation values in Asia & Oceania: Data heterogeneity and benefit transfer issues

We conduct a meta-analysis (MA) of around 100 studies valuing nature conservation in Asia and Oceania. Dividing our dataset into two levels of heterogeneity in terms of good characteristics (endangered species vs. nature conservation more generally) and valuation methods, we show that the degree of regularity and conformity with theory and empirical expectations is higher for the more homogenous dataset of contingent valuation of endangered species. For example, we find that willingness to pay (WTP) for preservation of mammals tends to be higher than other species and that WTP for species preservation increases with income. Increasing the degree of heterogeneity in the valuation data, however, preserves much of the regularity, and the explanatory power of some of our models is in the range of other MA studies of goods typically assumed to be more homogenous (such as water quality). Subjecting our best MA models to a simple test forecasting values for out-of-sample observations, shows median (mean) forecasting errors of 24 (46) percent for endangered species and 46 (89) percent for nature conservation more generally, approaching levels that may be acceptable in benefit transfer for policy use. We recommend that the most prudent MA practice is to control for heterogeneity in regressions and sensitivity analysis, rather than to limit datasets by non-transparent criteria to a level of heterogeneity deemed acceptable to the individual analyst. However, the trade-off will always be present and the issue of acceptable level of heterogeneity in MA is far from settle