Deep Learning based Recommender System: A Survey and New Perspectives

With the ever-growing volume of online information, recommender systems have been an effective strategy to overcome such information overload. The utility of recommender systems cannot be overstated, given its widespread adoption in many web applications, along with its potential impact to ameliorate many problems related to over-choice. In recent years, deep learning has garnered considerable interest in many research fields such as computer vision and natural language processing, owing not only to stellar performance but also the attractive property of learning feature representations from scratch. The influence of deep learning is also pervasive, recently demonstrating its effectiveness when applied to information retrieval and recommender systems research. Evidently, the field of deep learning in recommender system is flourishing. This article aims to provide a comprehensive review of recent research efforts on deep learning based recommender systems. More concretely, we provide and devise a taxonomy of deep learning based recommendation models, along with providing a comprehensive summary of the state-of-the-art. Finally, we expand on current trends and provide new perspectives pertaining to this new exciting development of the field.Comment: The paper has been accepted by ACM Computing Surveys. https://doi.acm.org/10.1145/328502

The Power of Linear Recurrent Neural Networks

Recurrent neural networks are a powerful means to cope with time series. We show how a type of linearly activated recurrent neural networks, which we call predictive neural networks, can approximate any time-dependent function f(t) given by a number of function values. The approximation can effectively be learned by simply solving a linear equation system; no backpropagation or similar methods are needed. Furthermore, the network size can be reduced by taking only most relevant components. Thus, in contrast to others, our approach not only learns network weights but also the network architecture. The networks have interesting properties: They end up in ellipse trajectories in the long run and allow the prediction of further values and compact representations of functions. We demonstrate this by several experiments, among them multiple superimposed oscillators (MSO), robotic soccer, and predicting stock prices. Predictive neural networks outperform the previous state-of-the-art for the MSO task with a minimal number of units.Comment: 22 pages, 14 figures and tables, revised implementatio

Modeling of complex nonlinear dynamic systems using temporal convolution neural networks

An increasingly important class of nonlinear systems includes the nonaffine hybrid systems, in particular those in which the underlying dynamics explicitly depends on a switching signal. When the inherent complexity is treatable and the phenomena governing the system dynamics are known an implicit model can be derived to describe its behaviour over time. Conversely, when these assumptions are not met the system dynamics can still be approximated by regression-based techniques, provided a dataset comprising inputs and outputs collected from the system is available. One approach to deal with data driven modelling relies on computational intelligent frameworks, in which artificial neural networks stand out as a prominent class of universal approximation black box models. This work aims to explore 1D Convolutional Neural Networks capabilities, in which the inputs are represented by regressors and structural configuration parameters, to modelling nonlinear hybrid dynamic systems. Moreover, in order evaluate the intrinsic ability to transparently approximate hybrid dynamics, this deep neural network architecture is compared to a shallow multilayer layer perceptron framework, in which each structural configuration is independently approximated.Uma classe de sistemas não lineares que tem vindo a ganhar cada vez mais importância é a dos sistemas híbridos não-afins, em particular aqueles em que a dinâmica subjacente depende explicitamente de um sinal de comutação. Quando a complexidade inerente é tratável e os fenómenos que controlam a dinâmica do sistema são conhecidos, é possível obter-se um modelo implícito para descrever seu comportamento ao longo do tempo. Por outro lado, quando essas suposições não são cumpridas, a dinâmica do sistema pode ainda ser aproximada por técnicas baseadas em regressão, desde que um conjunto de dados contendo as entradas e as saídas do sistema esteja disponível. Uma abordagem para lidar com o problema de modelação experimental recorrendo a técnicas de inteligência computacional, na quais as redes neuronais artificiais se destacam como uma das classes proeminentes de aproximadores universais. Este trabalho tem como objetivo explorar as capacidades de redes neuronais convolutivas 1D, onde as entradas são representadas por regressores e parâmetros de configuração estrutural. Além disso, para avaliar a capacidade intrínseca para a aproximação de dinâmicas híbridas, esta arquitetura de rede neuronal profunda é comparada a uma estrutura neuronal proactivas multicamada, na qual cada configuração estrutural é independentemente aproximada

Space-partitioning with cascade-connected ANN structures for positioning in mobile communication systems

The world around us is getting more connected with each day passing by – new portable devices employing wireless connections to various networks wherever one might be. Locationaware computing has become an important bit of telecommunication services and industry. For this reason, the research efforts on new and improved localisation algorithms are constantly being performed. Thus far, the satellite positioning systems have achieved highest popularity and penetration regarding the global position estimation. In spite the numerous investigations aimed at enabling these systems to equally procure the position in both indoor and outdoor environments, this is still a task to be completed. This research work presented herein aimed at improving the state-of-the-art positioning techniques through the use of two highly popular mobile communication systems: WLAN and public land mobile networks. These systems already have widely deployed network structures (coverage) and a vast number of (inexpensive) mobile clients, so using them for additional, positioning purposes is rational and logical. First, the positioning in WLAN systems was analysed and elaborated. The indoor test-bed, used for verifying the models’ performances, covered almost 10,000m2 area. It has been chosen carefully so that the positioning could be thoroughly explored. The measurement campaigns performed therein covered the whole of test-bed environment and gave insight into location dependent parameters available in WLAN networks. Further analysis of the data lead to developing of positioning models based on ANNs. The best single ANN model obtained 9.26m average distance error and 7.75m median distance error. The novel positioning model structure, consisting of cascade-connected ANNs, improved those results to 8.14m and 4.57m, respectively. To adequately compare the proposed techniques with other, well-known research techniques, the environment positioning error parameter was introduced. This parameter enables to take the size of the test environment into account when comparing the accuracy of the indoor positioning techniques. Concerning the PLMN positioning, in-depth analysis of available system parameters and signalling protocols produced a positioning algorithm, capable of fusing the system received signal strength parameters received from multiple systems and multiple operators. Knowing that most of the areas are covered by signals from more than one network operator and even more than one system from one operator, it becomes easy to note the great practical value of this novel algorithm. On the other hand, an extensive drive-test measurement campaign, covering more than 600km in the central areas of Belgrade, was performed. Using this algorithm and applying the single ANN models to the recorded measurements, a 59m average distance error and 50m median distance error were obtained. Moreover, the positioning in indoor environment was verified and the degradation of performances, due to the crossenvironment model use, was reported: 105m average distance error and 101m median distance error. When applying the new, cascade-connected ANN structure model, distance errors were reduced to 26m and 2m, for the average and median distance errors, respectively. The obtained positioning accuracy was shown to be good enough for the implementation of a broad scope of location based services by using the existing and deployed, commonly available, infrastructure

Predictive Accuracy of Recommender Algorithms

Recommender systems present a customized list of items based upon user or item characteristics with the objective of reducing a large number of possible choices to a smaller ranked set most likely to appeal to the user. A variety of algorithms for recommender systems have been developed and refined including applications of deep learning neural networks. Recent research reports point to a need to perform carefully controlled experiments to gain insights about the relative accuracy of different recommender algorithms, because studies evaluating different methods have not used a common set of benchmark data sets, baseline models, and evaluation metrics. The dissertation used publicly available sources of ratings data with a suite of three conventional recommender algorithms and two deep learning (DL) algorithms in controlled experiments to assess their comparative accuracy. Results for the non-DL algorithms conformed well to published results and benchmarks. The two DL algorithms did not perform as well and illuminated known challenges implementing DL recommender algorithms as reported in the literature. Model overfitting is discussed as a potential explanation for the weaker performance of the DL algorithms and several regularization strategies are reviewed as possible approaches to improve predictive error. Findings justify the need for further research in the use of deep learning models for recommender systems

Towards The Deep Semantic Learning Machine Neuroevolution Algorithm: An exploration on the CIFAR-10 problem task

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsSelecting the topology and parameters of Convolutional Neural Network (CNN) for a given supervised machine learning task is a non-trivial problem. The Deep Semantic Learning Machine (Deep-SLM) deals with this problem by automatically constructing CNNs without the use of the Backpropagation algorithm. The Deep-SLM is a novel neuroevolution technique and functions as stochastic semantic hill-climbing algorithm searching over the space of CNN topologies and parameters. The geometric semantic properties of the Deep-SLM induce a unimodel error space and eliminate the existence of local optimal solutions. This makes the Deep-SLM potentially favorable in terms of search efficiency and effectiveness. This thesis provides an exploration of a variant of the Deep-SLM algorithm on the CIFAR-10 problem task, and a validation of its proof of concept. This specific variant only forms mutation node ! mutation node connections in the non-convolutional part of the constructed CNNs. Furthermore, a comparative study between the Deep-SLM and the Semantic Learning Machine (SLM) algorithms was conducted. It was observed that sparse connections can be an effective way to prevent overfitting. Additionally, it was shown that a single 2D convolution layer initialized with random weights does not result in well-generalizing features for the Deep-SLM directly, but, in combination with a 2D max-pooling down sampling layer, effective improvements in performance and generalization of the Deep-SLM could be achieved. These results constitute to the hypothesis that convolution and pooling layers can improve performance and generalization of the Deep-SLM, unless the components are properly optimized.Selecionar a topologia e os parâmetros da Rede Neural Convolucional (CNN) para uma tarefa de aprendizado automático supervisionada não é um problema trivial. A Deep Semantic Learning Machine (Deep-SLM) lida com este problema construindo automaticamente CNNs sem recorrer ao uso do algoritmo de Retro-propagação. A Deep-SLM é uma nova técnica de neuroevolução que funciona enquanto um algoritmo de escalada estocástico semântico na pesquisa de topologias e de parâmetros CNN. As propriedades geométrico-semânticas da Deep-SLM induzem um unimodel error space que elimina a existência de soluções ótimas locais, favorecendo, potencialmente, a Deep-SLM em termos de eficiência e eficácia. Esta tese providencia uma exploração de uma variante do algoritmo da Deep-SLM no problemo de CIFAR-10, assim como uma validação do seu conceito de prova. Esta variante específica apenas forma conexões nó de mutação!nó de mutação na parte non convolucional da CNN construída. Mais ainda, foi conduzido um estudo comparativo entre a Deep-SLM e o algoritmo da Semantic Learning Machine (SLM). Tendo sido observado que as conexões esparsas poderão tratar-se de uma forma eficiente de prevenir o overfitting. Adicionalmente, mostrou-se que uma singular camada de convolução 2D, iniciada com valores aleatórios, não resulta, directamente, em características generalizadas para a Deep-SLM, mas, em combinação com uma camada de 2D max-pooling, melhorias efectivas na performance e na generalização da Deep-SLM poderão ser concretizadas. Estes resultados constituem, assim, a hipótese de que as camadas de convolução e pooling poderão melhorar a performance e a generalização da Deep-SLM, a não ser que os componentes sejam adequadamente otimizados