Deep Collaborative Learning for Randomly Wired Neural Networks

A deep collaborative learning approach is introduced in which a chain of randomly wired neural networks is trained simultaneously to improve the overall generalization and form a strong ensemble model. The proposed method takes advantage of functional-preserving transfer learning and knowledge distillation to produce an ensemble model. Knowledge distillation is an effective learning scheme for improving the performance of small neural networks by using the knowledge learned by teacher networks. Most of the previous methods learn from one or more teachers but not in a collaborative way. In this paper, we created a chain of randomly wired neural networks based on a random graph algorithm and collaboratively trained the models using functional-preserving transfer learning, so that the small network in the chain could learn from the largest one simultaneously. The training method applies knowledge distillation between randomly wired models, where each model is considered as a teacher to the next model in the chain. The decision of multiple chains of models can be combined to produce a robust ensemble model. The proposed method is evaluated on CIFAR-10, CIFAR-100, and TinyImageNet. The experimental results show that the collaborative training significantly improved the generalization of each model, which allowed for obtaining a small model that can mimic the performance of a large model and produce a more robust ensemble approach

Transfer Meets Hybrid: A Synthetic Approach for Cross-Domain Collaborative Filtering with Text

Collaborative filtering (CF) is the key technique for recommender systems (RSs). CF exploits user-item behavior interactions (e.g., clicks) only and hence suffers from the data sparsity issue. One research thread is to integrate auxiliary information such as product reviews and news titles, leading to hybrid filtering methods. Another thread is to transfer knowledge from other source domains such as improving the movie recommendation with the knowledge from the book domain, leading to transfer learning methods. In real-world life, no single service can satisfy a user's all information needs. Thus it motivates us to exploit both auxiliary and source information for RSs in this paper. We propose a novel neural model to smoothly enable Transfer Meeting Hybrid (TMH) methods for cross-domain recommendation with unstructured text in an end-to-end manner. TMH attentively extracts useful content from unstructured text via a memory module and selectively transfers knowledge from a source domain via a transfer network. On two real-world datasets, TMH shows better performance in terms of three ranking metrics by comparing with various baselines. We conduct thorough analyses to understand how the text content and transferred knowledge help the proposed model.Comment: 11 pages, 7 figures, a full version for the WWW 2019 short pape

Deep Learning based Recommender System: A Survey and New Perspectives

With the ever-growing volume of online information, recommender systems have been an effective strategy to overcome such information overload. The utility of recommender systems cannot be overstated, given its widespread adoption in many web applications, along with its potential impact to ameliorate many problems related to over-choice. In recent years, deep learning has garnered considerable interest in many research fields such as computer vision and natural language processing, owing not only to stellar performance but also the attractive property of learning feature representations from scratch. The influence of deep learning is also pervasive, recently demonstrating its effectiveness when applied to information retrieval and recommender systems research. Evidently, the field of deep learning in recommender system is flourishing. This article aims to provide a comprehensive review of recent research efforts on deep learning based recommender systems. More concretely, we provide and devise a taxonomy of deep learning based recommendation models, along with providing a comprehensive summary of the state-of-the-art. Finally, we expand on current trends and provide new perspectives pertaining to this new exciting development of the field.Comment: The paper has been accepted by ACM Computing Surveys. https://doi.acm.org/10.1145/328502

Learning from Physics Education Research: Lessons for Economics Education

We believe that economists have much to learn from educational research practices and related pedagogical innovations in other disciplines, in particular physics education. In this paper we identify three key features of physics education research that distinguish it from economics education research - (1) the intentional grounding of physics education research in learning science principles, (2) a shared conceptual research framework focused on how students learn physics concepts, and (3) a cumulative process of knowledge-building in the discipline - and describe their influence on new teaching pedagogies, instructional activities, and curricular design in physics education. In addition, we highlight four specific examples of successful pedagogical innovations drawn from physics education - context-rich problems, concept tests, just-in-time teaching, and interactive lecture demonstrations - and illustrate how these practices can be adapted for economic education.Comment: 19 pages, 3 figures, submitted to Journal of Economic Education, also available from Social Science Research Network <http://ssrn.com/abstract=1151430

Online discussion compensates for suboptimal timing of supportive information presentation in a digitally supported learning environment

This study used a sequential set-up to investigate the consecutive effects of timing of supportive information presentation (information before vs. information during the learning task clusters) in interactive digital learning materials (IDLMs) and type of collaboration (personal discussion vs. online discussion) in computer-supported collaborative learning (CSCL) on student knowledge construction. Students (N = 87) were first randomly assigned to the two information presentation conditions to work individually on a case-based assignment in IDLM. Students who received information during learning task clusters tended to show better results on knowledge construction than those who received information only before each cluster. The students within the two separate information presentation conditions were then randomly assigned to pairs to discuss the outcomes of their assignments under either the personal discussion or online discussion condition in CSCL. When supportive information had been presented before each learning task cluster, online discussion led to better results than personal discussion. When supportive information had been presented during the learning task clusters, however, the online and personal discussion conditions had no differential effect on knowledge construction. Online discussion in CSCL appeared to compensate for suboptimal timing of presentation of supportive information before the learning task clusters in IDLM

Multi-Target Prediction: A Unifying View on Problems and Methods

Multi-target prediction (MTP) is concerned with the simultaneous prediction of multiple target variables of diverse type. Due to its enormous application potential, it has developed into an active and rapidly expanding research field that combines several subfields of machine learning, including multivariate regression, multi-label classification, multi-task learning, dyadic prediction, zero-shot learning, network inference, and matrix completion. In this paper, we present a unifying view on MTP problems and methods. First, we formally discuss commonalities and differences between existing MTP problems. To this end, we introduce a general framework that covers the above subfields as special cases. As a second contribution, we provide a structured overview of MTP methods. This is accomplished by identifying a number of key properties, which distinguish such methods and determine their suitability for different types of problems. Finally, we also discuss a few challenges for future research