Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon

How to develop slim and accurate deep neural networks has become crucial for real- world applications, especially for those employed in embedded systems. Though previous work along this research line has shown some promising results, most existing methods either fail to significantly compress a well-trained deep network or require a heavy retraining process for the pruned deep network to re-boost its prediction performance. In this paper, we propose a new layer-wise pruning method for deep neural networks. In our proposed method, parameters of each individual layer are pruned independently based on second order derivatives of a layer-wise error function with respect to the corresponding parameters. We prove that the final prediction performance drop after pruning is bounded by a linear combination of the reconstructed errors caused at each layer. Therefore, there is a guarantee that one only needs to perform a light retraining process on the pruned network to resume its original prediction performance. We conduct extensive experiments on benchmark datasets to demonstrate the effectiveness of our pruning method compared with several state-of-the-art baseline methods

Distributed Multi-Task Relationship Learning

Multi-task learning aims to learn multiple tasks jointly by exploiting their relatedness to improve the generalization performance for each task. Traditionally, to perform multi-task learning, one needs to centralize data from all the tasks to a single machine. However, in many real-world applications, data of different tasks may be geo-distributed over different local machines. Due to heavy communication caused by transmitting the data and the issue of data privacy and security, it is impossible to send data of different task to a master machine to perform multi-task learning. Therefore, in this paper, we propose a distributed multi-task learning framework that simultaneously learns predictive models for each task as well as task relationships between tasks alternatingly in the parameter server paradigm. In our framework, we first offer a general dual form for a family of regularized multi-task relationship learning methods. Subsequently, we propose a communication-efficient primal-dual distributed optimization algorithm to solve the dual problem by carefully designing local subproblems to make the dual problem decomposable. Moreover, we provide a theoretical convergence analysis for the proposed algorithm, which is specific for distributed multi-task relationship learning. We conduct extensive experiments on both synthetic and real-world datasets to evaluate our proposed framework in terms of effectiveness and convergence.Comment: To appear in KDD 201

Hashing over Predicted Future Frames for Informed Exploration of Deep Reinforcement Learning

In deep reinforcement learning (RL) tasks, an efficient exploration mechanism should be able to encourage an agent to take actions that lead to less frequent states which may yield higher accumulative future return. However, both knowing about the future and evaluating the frequentness of states are non-trivial tasks, especially for deep RL domains, where a state is represented by high-dimensional image frames. In this paper, we propose a novel informed exploration framework for deep RL, where we build the capability for an RL agent to predict over the future transitions and evaluate the frequentness for the predicted future frames in a meaningful manner. To this end, we train a deep prediction model to predict future frames given a state-action pair, and a convolutional autoencoder model to hash over the seen frames. In addition, to utilize the counts derived from the seen frames to evaluate the frequentness for the predicted frames, we tackle the challenge of matching the predicted future frames and their corresponding seen frames at the latent feature level. In this way, we derive a reliable metric for evaluating the novelty of the future direction pointed by each action, and hence inform the agent to explore the least frequent one

The Lattice Kinetic Monte Carlo Method as a Versatile Tool for Simulating Diverse Micro and Mesoscale Phenomena

Lattice-based, or ‘on-lattice’, kinetic Monte Carlo simulations are attractive because of their relative computational simplicity and efficiency, and have been employed to simulate an enormous range of non-equilibrium physical, chemical and biological phenomena. In a kinetic Monte Carlo simulation (also referred to as dynamic Monte Carlo or the Gillespie method), which is typically applied to a collection of discrete particles, rates must first be specified for a set of ‘events’, such as a hop of an atom from one lattice site to another or a reaction between two particles. The universe of events and the associated rates are input to the kinetic Monte Carlo simulation, and must be obtained by some other means. The simulation is then carried out by executing events in a biased stochastic sequence. In this talk, variants of the lattice kinetic Monte Carlo method are applied to several distinct situations, ranging from microstructure evolution in semiconductor crystals, to cellular aggregate formation in blood flow, to coarse-grained simulations of phase evolution in generalized liquid-vapor systems. These diverse examples are used to illustrate the simplicity and flexibility of the general lattice kinetic Monte Carlo framework as a powerful computational tool, but also the potential pitfalls related to its application in certain situations. In this regard, special emphasis is placed on the discussion of (1) reduced degree-of-freedom representations (via coarse-graining) and the concomitant loss of entropy, and (2) simulation of systems of particles subject to external advective fields such as fluid flow

Managing Cultural Diversity: The Case of Small & Medium Tourism Enterprises (SMTE)

For a long time, management has been developing and applying a standardized universal approach with its employees to optimize the operation of companies. This is regardless of the company’s nature and the country where it operates. The failure of certain firms, the increasing globalization of trade, the circulation of goods, and the fast transfer of individuals and information have led to the need to take into account other aspects of management, in particular the intercultural factor. Today, employees from different cultural backgrounds are working as a team in the same company. Nevertheless, differences in understanding values and visions of the world can create intra-team misunderstandings in collaborative work. Managing cultural diversity reduces these misunderstandings so that the benefits of the diversity emerge. Managing cultural diversity means managing opposing opinions, constant contradictions, continuous oppositions, and different perceptions. Furthermore, managing cultural diversity contributes to improving team effectiveness where diverse teams often perform better than homogenous teams in problem solving and complex tasks. Greater diversity leads to innovative, higher-quality decisions, and solutions. This paper sheds light on the importance of cultural diversity among the company’s human resources and the role of cultural intelligence in intercultural management. As part of the research, a study centered on the Lebanese small and medium tourism enterprises (SMTE) was conducted. The results obtained showed a shy diversity management approach in the Lebanese tourism sector due to several factors, namely: the economic instability and lack of training

Morphine more fine? Its effects in critically ill newborns

The pharmacist Sertürner first isolated morphine from opium in 1803 and named it after Morpheus, the god of dreams in Greco-Roman mythology. Ever since, it has been one of the most frequently used drugs to relieve pain, for a variety of age groups. In our days, however, there is still debate whether morphine and analgesic therapy should serve as standard of care for hospitalized newborns. Until the last decade of the 20th century, premature neonates were generally believed to have little pain sensation and thus not in need of analgesic therapy. The studies of Anand et al., which showed decreased morbidity and mortality in neonatal patients receiving adequate analgesic therapy after surgery, were instrumental in altering this notion. At present it is widely recognized that even the most premature neonates can feel pain. While neonatal pain experiences have been suggested to bring about short and long-term negative consequences, analgesic therapy in the vulnerable newborns can also carry risks, such as increased incidence of seizures. This leaves us with the question whether the benefits of treatment will outweigh the side effects and potential hazards of analgesic treatment. Or, in other words, is morphine more fine? The studies described in this thesis generally aim to improve neonatal pain treatment, by investigating the beneficial and adverse effects of neonatal morphine use. They also aim at improving our knowledge of how newborns respond to pain and how to measure pain objectively: pain assessment