Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Data

The implicit bias towards solutions with favorable properties is believed to be a key reason why neural networks trained by gradient-based optimization can generalize well. While the implicit bias of gradient flow has been widely studied for homogeneous neural networks (including ReLU and leaky ReLU networks), the implicit bias of gradient descent is currently only understood for smooth neural networks. Therefore, implicit bias in non-smooth neural networks trained by gradient descent remains an open question. In this paper, we aim to answer this question by studying the implicit bias of gradient descent for training two-layer fully connected (leaky) ReLU neural networks. We showed that when the training data are nearly-orthogonal, for leaky ReLU activation function, gradient descent will find a network with a stable rank that converges to $1$, whereas for ReLU activation function, gradient descent will find a neural network with a stable rank that is upper bounded by a constant. Additionally, we show that gradient descent will find a neural network such that all the training data points have the same normalized margin asymptotically. Experiments on both synthetic and real data backup our theoretical findings.Comment: 55 pages, 7 figures. In NeurIPS 202

Reply to Komatsu et al.: From local social mindfulness to global sustainability efforts?

Komatsu et al. (1) argue that Van Doesum et al. (2) may have overlooked the role of GDP in reporting a positive association between social mindfulness (SoMi) and the Environmental Performance Index (EPI) at country level. Although the relationship between EPI and SoMi is relatively weaker for countries with higher GDP, that does not imply that the overall observed relationship is a statistical artifact. Rather, it implies that GDP may be a moderator of the relationship between EPI and SoMi. The observed correlation is a valid result on average across countries, and the actual effect size would, at least to some degree, depend on GDP

Reply to Nielsen et al.: Social mindfulness is associated with countries' environmental performance and individual environmental concern.

Nielsen et al. (1) argue that Van Doesum et al. (2) need to consider three points for their interpretation of a positive association between individual-level social mindfulness (SoMi) and environmental performance (EPI) at the country level (3). The association is weaker when 1) it is controlled for GDP and 2) when the data of three countries are removed; also, 3) the data do not address the association between SoMi and individual-level environmental concern. We discuss these points in turn

Reply to Nielsen et al. social mindfulness is associated with countries’ environmental performance and individual environmental concern

info:eu-repo/semantics/publishedVersio

From local social mindfulness to global sustainability efforts?

info:eu-repo/semantics/publishedVersio

Social mindfulness and prosociality vary across the globe

Humans are social animals, but not everyone will be mindful of others to the same extent. Individual differences have been found, but would social mindfulness also be shaped by one’s location in the world? Expecting cross-national differences to exist, we examined if and how social mindfulness differs across countries. At little to no material cost, social mindfulness typically entails small acts of attention or kindness. Even though fairly common, such low-cost cooperation has received little empirical attention. Measuring social mindfulness across 31 samples from industrialized countries and regions (n = 8,354), we found considerable variation. Among selected country-level variables, greater social mindfulness was most strongly associated with countries’ better general performance on environmental protection. Together, our findings contribute to the literature on prosociality by targeting the kind of everyday cooperation that is more focused on communicating benevolence than on providing material benefits

"As Uploaders, We Have the Responsibility”: Individualized Professionalization of Bilibili Uploaders

The prevalence of social media blurs the boundaries between consumer and producer, work and play, and leads to new social roles, professions, and identities (e.g. blogger, YouTuber, micro-celebrity). However, we still lack a clear understanding of how people come to identify with these new roles and how individual professional development is digitally mediated. This paper presents a study based on Bilibili, a popular Chinese social media platform featuring user-generated videos, and highlights a professionalization process through which individuals consciously distinguish between the roles of uploaders and consumers, develop a shared work ethos around the role of the uploader, and, as uploaders, improve their technical-professional expertise. We conclude by discussing individualized professionalization as a concept that describes the bottom-up and community-based process of professional development for User Generated Content (UGC) taking place in contemporary digital media environments

The moss genus Didymodon as an indicator of climate change on the Tibetan Plateau

Bryophytes are sensitive to changing atmospheric conditions. Potentially, typical alpine bryophytes are valuable indicators of climate change for alpine ecosystems. However, little is known about the effects of climate change on the dominant alpine bryophytes of the Tibetan Plateau. In this study, we compared suitable habitats and predicted the effects of key environmental variables, in three climate change scenarios (RCP 2.6, RCP 4.5 and RCP 8.5), on the distribution of two dominant moss genera, a typically alpine xerophytic functional group, Bryoerythrophyllum and Didymodon in Pottiaceae. We used Maximum Entropy model (MaxEnt) modelling, for the 2050s and 2070s in Tibet. Simulation-based estimates suggest that Bryoerythrophyllum are more suited to habitats from semi-humid regions to semi-arid regions while Didymodon are relatively drought-resistant mosses that mainly inhabit drought regions. The variables associated with temperature will have the strongest effect on the future distribution patterns of both Bryoerythrophyllum and Didymodon when compared with precipitation and topographic variables. However, Climate change should affect Didymodon the most, due to its relatively narrow temperature and precipitation range for optimum growth. Additionally, Bryoerythrophyllum is predicted with a ratio of increase in suitable areas at 19.08%, 141.49% and 121.56% and 55.22%, 129.93% and 172.08% under all three scenarios, for 2050 and 2070, respectively. In contrast, Didymodon has a ratio of increase in suitable areas at −25.36%, 12.51% and 51.1% and 13.71%, 14.27% and 131.91%, for all three scenarios for the two future decades, respectively. However, there are more obviously regular expansions and contractions can be easily caught for Didymodon than for Bryoerythrophyllum with GHG emissions from low to high. In summary, although Bryoerythrophyllum and Didymodon are evolutionarily closely related and have similar distribution patterns and future upward and northward shifts, Didymodon will respond more to climate warming under each climatic scenario for the 2050s and 2070s. Therefore, Didymodon is recommended as an indicator of climate change on the Tibetan Plateau