17,626 research outputs found
Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks
Empirical studies show that gradient-based methods can learn deep neural
networks (DNNs) with very good generalization performance in the
over-parameterization regime, where DNNs can easily fit a random labeling of
the training data. Very recently, a line of work explains in theory that with
over-parameterization and proper random initialization, gradient-based methods
can find the global minima of the training loss for DNNs. However, existing
generalization error bounds are unable to explain the good generalization
performance of over-parameterized DNNs. The major limitation of most existing
generalization bounds is that they are based on uniform convergence and are
independent of the training algorithm. In this work, we derive an
algorithm-dependent generalization error bound for deep ReLU networks, and show
that under certain assumptions on the data distribution, gradient descent (GD)
with proper random initialization is able to train a sufficiently
over-parameterized DNN to achieve arbitrarily small generalization error. Our
work sheds light on explaining the good generalization performance of
over-parameterized deep neural networks.Comment: 27 pages. This version simplifies the proof and improves the
presentation in Version 3. In AAAI 202
Unified framework for quantumness -- coherence, discord, and entanglement
From an operational perspective, quantumness characterizes the exotic
behavior in a physical process which cannot be explained with Newtonian
physics. There are several widely used measures of quantumness, including
coherence, discord, and entanglement, each proven to be essential resources in
particular situations. There exists evidence of fundamental connections amongst
the three measures. However, those quantumnesses are still regarded differently
and such connections are yet to be elucidated. Here, we introduce a general
framework of defining a unified quantumness with an operational motivation
founded on the capability of interferometry. The quantumness appears
differently as coherence, discord, and entanglement in different scenarios with
local measurement, weak reference frame free measurement, and strong reference
frame free measurement, respectively. Our results also elaborate how these
three measures are related and how they can be transformed from each other.
This framework can be further extended to other scenarios and serves as a
universal quantumness measure.Comment: 9 pages, 4 figure
Roads, Institutions and the Primary Sector in West Africa
West Africa is a fast growing, less studied region constrained by limited infrastructure facilities. This region faces increasing challenges of food security, disease and climate-reliant agriculture. This dissertation goes in depth into the region and explores the role of the basic infrastructure—roads—in various aspects, such as the resource allocations of governments and the impact of democratization, dynamics in regional connections, child health and climate resilience. The last essay estimates the impact of investment risks on mining sector in developing countries
- …