Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks

Empirical studies show that gradient-based methods can learn deep neural networks (DNNs) with very good generalization performance in the over-parameterization regime, where DNNs can easily fit a random labeling of the training data. Very recently, a line of work explains in theory that with over-parameterization and proper random initialization, gradient-based methods can find the global minima of the training loss for DNNs. However, existing generalization error bounds are unable to explain the good generalization performance of over-parameterized DNNs. The major limitation of most existing generalization bounds is that they are based on uniform convergence and are independent of the training algorithm. In this work, we derive an algorithm-dependent generalization error bound for deep ReLU networks, and show that under certain assumptions on the data distribution, gradient descent (GD) with proper random initialization is able to train a sufficiently over-parameterized DNN to achieve arbitrarily small generalization error. Our work sheds light on explaining the good generalization performance of over-parameterized deep neural networks.Comment: 27 pages. This version simplifies the proof and improves the presentation in Version 3. In AAAI 202

Roads, Institutions and the Primary Sector in West Africa

West Africa is a fast growing, less studied region constrained by limited infrastructure facilities. This region faces increasing challenges of food security, disease and climate-reliant agriculture. This dissertation goes in depth into the region and explores the role of the basic infrastructure—roads—in various aspects, such as the resource allocations of governments and the impact of democratization, dynamics in regional connections, child health and climate resilience. The last essay estimates the impact of investment risks on mining sector in developing countries

A case study of the meanings and values of educational research for participants in a Shanghai primary school and a Yorkshire primary school

This is a case study of teaching practitioners in two school settings, one in Shanghai and the other in the Yorkshire region of England. Its focus is on their views and values regarding the use of educational research in relation to their own practice and how their social practices as well as cultural orientations influence their use of different kinds of knowledge. The research problem addressed is the one identified in the 1990s’ policy debate about why much educational research does not seem to be directly helping teachers in improving their practice. Although the situation might have changed more recently with the greater development of evidence-based practice and some teachers’ active engagement in research, how practitioners make research meaningful to them remains unclear. This study aims to clarify how these teachers regard the idea of research-informed practice. The principal method of data collection was semi-structured interviewing. Other methods were used only to support the validity of interpretations in the analysis of interview data. This analysis shows that with respect to published research produced elsewhere and to their own research activities, the use, if any, that these practitioners make of research is likely to depend on whether they can appropriately recontextualize it according to the professional knowledge they value in their own networks or communities of practice. That process of recontextualization is one with which they are familiar from their social practice of transforming explicit codified knowledge embedded in curriculum documents, textbooks and other context-independent learning resources into pedagogic activities according to their tacit experiential understanding of what works in their particular situation. The high value placed on tacit situated knowledge as an essential component of professional knowledge and development is maintained when these practitioners extend recontextualization to research knowledge and evidence. It is present in the sense of agency and professional identity that is a priority for practitioners in each setting, although the respective cultural orientations towards it are different. A relational model is developed from the case study findings, with implications for revising the aims of educational research towards working with teachers to understand better their recontextualizing practice rather than seeking transferable prescriptions of pedagogy as a technical instrument

Unified framework for quantumness -- coherence, discord, and entanglement

From an operational perspective, quantumness characterizes the exotic behavior in a physical process which cannot be explained with Newtonian physics. There are several widely used measures of quantumness, including coherence, discord, and entanglement, each proven to be essential resources in particular situations. There exists evidence of fundamental connections amongst the three measures. However, those quantumnesses are still regarded differently and such connections are yet to be elucidated. Here, we introduce a general framework of defining a unified quantumness with an operational motivation founded on the capability of interferometry. The quantumness appears differently as coherence, discord, and entanglement in different scenarios with local measurement, weak reference frame free measurement, and strong reference frame free measurement, respectively. Our results also elaborate how these three measures are related and how they can be transformed from each other. This framework can be further extended to other scenarios and serves as a universal quantumness measure.Comment: 9 pages, 4 figure