Progressive Label Distillation: Learning Input-Efficient Deep Neural Networks

Much of the focus in the area of knowledge distillation has been on distilling knowledge from a larger teacher network to a smaller student network. However, there has been little research on how the concept of distillation can be leveraged to distill the knowledge encapsulated in the training data itself into a reduced form. In this study, we explore the concept of progressive label distillation, where we leverage a series of teacher-student network pairs to progressively generate distilled training data for learning deep neural networks with greatly reduced input dimensions. To investigate the efficacy of the proposed progressive label distillation approach, we experimented with learning a deep limited vocabulary speech recognition network based on generated 500ms input utterances distilled progressively from 1000ms source training data, and demonstrated a significant increase in test accuracy of almost 78% compared to direct learning.Comment: 9 page

A priori and a posteriori W1,∞W^{1,\infty} error analysis of a QC method for complex lattices

In this paper we prove a priori and a posteriori error estimates for a multiscale numerical method for computing equilibria of multilattices under an external force. The error estimates are derived in a $W^{1,\infty}$ norm in one space dimension. One of the features of our analysis is that we establish an equivalent way of formulating the coarse-grained problem which greatly simplifies derivation of the error bounds (both, a priori and a posteriori). We illustrate our error estimates with numerical experiments.Comment: 23 page

Towards Prediction of Non-Radiative Decay Pathways in Organic Compounds I: The Case of Naphthalene Quantum Yields

Many emerging technologies depend on human’s ability to control and manipulate the excited-state properties of molecular systems. These technologies include fluorescent labeling in biomedical imaging, light harvesting in photovoltaics, and electroluminescence in light-emitting devices. All of these systems suffer from non-radiative loss pathways that dissipate electronic energy as heat, which causes the overall system efficiency to be directly linked to quantum yield (Φ) of the molecular excited state. Unfortunately, Φ is very difficult to predict from first principles because the description of a slow non-radiative decay mechanism requires an accurate description of long-timescale excited-state quantum dynamics. In the present study, we introduce an efficient semiempirical method of calculating the fluorescence quantum yield (Φfl) for molecular chromophores, which, based on machine learning, converts simple electronic energies computed using time-dependent density functional theory (TDDFT) into an estimate of Φfl. As with all machine learning strategies, the algorithm needs to be trained on fluorescent dyes for which Φfl’s are known, so as to provide a black-box method which can later predict Φfl’s for chemically similar chromophores that have not been studied experimentally. As a first illustration of how our proposed algorithm can be trained, we examine a family of 25 naphthalene derivatives. The simplest application of the energy gap law is found to be inadequate to explain the rates of internal conversion (IC) or intersystem crossing (ISC) – the electronic properties of at least one higher-lying electronic state (Sn or Tn) or one far-from-equilibrium geometry are typically needed to obtain accurate results. Indeed, the key descriptors turn out to be the transition state between the Franck–Condon minimum a distorted local minimum near an S0/S1 conical intersection (which governs IC) and the magnitude of the spin–orbit coupling (which governs ISC). The resulting Φfl’s are predicted with reasonable accuracy (±22%), making our approach a promising ingredient for high-throughput screening and rational design of the molecular excited states with desired Φ’s. We thus conclude that our model, while semi-empirical in nature, does in fact extract sound physical insight into the challenge of describing non-radiative relaxations