49 research outputs found
Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Data
The implicit bias towards solutions with favorable properties is believed to
be a key reason why neural networks trained by gradient-based optimization can
generalize well. While the implicit bias of gradient flow has been widely
studied for homogeneous neural networks (including ReLU and leaky ReLU
networks), the implicit bias of gradient descent is currently only understood
for smooth neural networks. Therefore, implicit bias in non-smooth neural
networks trained by gradient descent remains an open question. In this paper,
we aim to answer this question by studying the implicit bias of gradient
descent for training two-layer fully connected (leaky) ReLU neural networks. We
showed that when the training data are nearly-orthogonal, for leaky ReLU
activation function, gradient descent will find a network with a stable rank
that converges to , whereas for ReLU activation function, gradient descent
will find a neural network with a stable rank that is upper bounded by a
constant. Additionally, we show that gradient descent will find a neural
network such that all the training data points have the same normalized margin
asymptotically. Experiments on both synthetic and real data backup our
theoretical findings.Comment: 55 pages, 7 figures. In NeurIPS 202
Reply to Komatsu et al.: From local social mindfulness to global sustainability efforts?
Komatsu et al. (1) argue that Van Doesum et al. (2)
may have overlooked the role of GDP in reporting a
positive association between social mindfulness
(SoMi) and the Environmental Performance Index
(EPI) at country level. Although the relationship
between EPI and SoMi is relatively weaker for countries with higher GDP, that does not imply that the
overall observed relationship is a statistical artifact.
Rather, it implies that GDP may be a moderator of
the relationship between EPI and SoMi. The
observed correlation is a valid result on average
across countries, and the actual effect size would, at
least to some degree, depend on GDP
Reply to Nielsen et al.: Social mindfulness is associated with countries' environmental performance and individual environmental concern.
Nielsen et al. (1) argue that Van Doesum et al. (2)
need to consider three points for their interpretation
of a positive association between individual-level
social mindfulness (SoMi) and environmental performance (EPI) at the country level (3). The association
is weaker when 1) it is controlled for GDP and 2)
when the data of three countries are removed; also,
3) the data do not address the association between
SoMi and individual-level environmental concern.
We discuss these points in turn
Reply to Nielsen et al. social mindfulness is associated with countriesâ environmental performance and individual environmental concern
info:eu-repo/semantics/publishedVersio
From local social mindfulness to global sustainability efforts?
info:eu-repo/semantics/publishedVersio
Social mindfulness and prosociality vary across the globe
Humans are social animals, but not everyone will be mindful of others to the same extent. Individual differences have been found, but would social mindfulness also be shaped by oneâs location in the world? Expecting cross-national differences to exist, we examined if and how social mindfulness differs across countries. At little to no material cost, social mindfulness typically entails small acts of attention or kindness. Even though fairly common, such low-cost cooperation has received little empirical attention. Measuring social mindfulness across 31 samples from industrialized countries and regions (n = 8,354), we found considerable variation. Among selected country-level variables, greater social mindfulness was most strongly associated with countriesâ better general performance on environmental protection. Together, our findings contribute to the literature on prosociality by targeting the kind of everyday cooperation that is more focused on communicating benevolence than on providing material benefits
"As Uploaders, We Have the Responsibilityâ: Individualized Professionalization of Bilibili Uploaders
The prevalence of social media blurs the boundaries between consumer and producer, work and play, and leads to new social roles, professions, and identities (e.g. blogger, YouTuber, micro-celebrity). However, we still lack a clear understanding of how people come to identify with these new roles and how individual professional development is digitally mediated. This paper presents a study based on Bilibili, a popular Chinese social media platform featuring user-generated videos, and highlights a professionalization process through which individuals consciously distinguish between the roles of uploaders and consumers, develop a shared work ethos around the role of the uploader, and, as uploaders, improve their technical-professional expertise. We conclude by discussing individualized professionalization as a concept that describes the bottom-up and community-based process of professional development for User Generated Content (UGC) taking place in contemporary digital media environments
The moss genus Didymodon as an indicator of climate change on the Tibetan Plateau
Bryophytes are sensitive to changing atmospheric conditions. Potentially, typical alpine bryophytes are valuable indicators of climate change for alpine ecosystems. However, little is known about the effects of climate change on the dominant alpine bryophytes of the Tibetan Plateau. In this study, we compared suitable habitats and predicted the effects of key environmental variables, in three climate change scenarios (RCP 2.6, RCP 4.5 and RCP 8.5), on the distribution of two dominant moss genera, a typically alpine xerophytic functional group, Bryoerythrophyllum and Didymodon in Pottiaceae. We used Maximum Entropy model (MaxEnt) modelling, for the 2050s and 2070s in Tibet. Simulation-based estimates suggest that Bryoerythrophyllum are more suited to habitats from semi-humid regions to semi-arid regions while Didymodon are relatively drought-resistant mosses that mainly inhabit drought regions. The variables associated with temperature will have the strongest effect on the future distribution patterns of both Bryoerythrophyllum and Didymodon when compared with precipitation and topographic variables. However, Climate change should affect Didymodon the most, due to its relatively narrow temperature and precipitation range for optimum growth. Additionally, Bryoerythrophyllum is predicted with a ratio of increase in suitable areas at 19.08%, 141.49% and 121.56% and 55.22%, 129.93% and 172.08% under all three scenarios, for 2050 and 2070, respectively. In contrast, Didymodon has a ratio of increase in suitable areas at â25.36%, 12.51% and 51.1% and 13.71%, 14.27% and 131.91%, for all three scenarios for the two future decades, respectively. However, there are more obviously regular expansions and contractions can be easily caught for Didymodon than for Bryoerythrophyllum with GHG emissions from low to high. In summary, although Bryoerythrophyllum and Didymodon are evolutionarily closely related and have similar distribution patterns and future upward and northward shifts, Didymodon will respond more to climate warming under each climatic scenario for the 2050s and 2070s. Therefore, Didymodon is recommended as an indicator of climate change on the Tibetan Plateau