476 research outputs found

    Recommender Systems in Light of Big Data

    Get PDF
    The growth in the usage of the web, especially e-commerce website, has led to the development of recommender system (RS) which aims in personalizing the web content for each user and reducing the cognitive load of information on the user. However, as the world enters Big Data era and lives through the contemporary data explosion, the main goal of a RS becomes to provide millions of high quality recommendations in few seconds for the increasing number of users and items. One of the successful techniques of RSs is collaborative filtering (CF) which makes recommendations for users based on what other like-mind users had preferred. Despite its success, CF is facing some challenges posed by Big Data, such as: scalability, sparsity and cold start. As a consequence, new approaches of CF that overcome the existing problems have been studied such as Singular value decomposition (SVD). This paper surveys the literature of RSs and reviews the current state of RSs with the main concerns surrounding them due to Big Data. Furthermore, it investigates thoroughly SVD, one of the promising approaches expected to perform well in tackling Big Data challenges, and provides an implementation to it using some of the successful Big Data tools (i.e. Apache Hadoop and Spark). This implementation is intended to validate the applicability of, existing contributions to the field of, SVD-based RSs as well as validated the effectiveness of Hadoop and spark in developing large-scale systems. The implementation has been evaluated empirically by measuring mean absolute error which gave comparable results with other experiments conducted, previously by other researchers, on a relatively smaller data set and non-distributed environment. This proved the scalability of SVD-based RS and its applicability to Big Data

    Improving Online Education Using Big Data Technologies

    Get PDF
    In a world in full digital transformation, where new information and communication technologies are constantly evolving, the current challenge of Computing Environments for Human Learning (CEHL) is to search the right way to integrate and harness the power of these technologies. In fact, these environments face many challenges, especially the increased demand for learning, the huge growth in the number of learners, the heterogeneity of available resources as well as the problems related to the complexity of intensive processing and real-time analysis of data produced by e-learning systems, which goes beyond the limits of traditional infrastructures and relational database management systems. This chapter presents a number of solutions dedicated to CEHL around the two big paradigms, namely cloud computing and Big Data. The first part of this work is dedicated to the presentation of an approach to integrate both emerging technologies of the big data ecosystem and on-demand services of the cloud in the e-learning field. It aims to enrich and enhance the quality of e-learning platforms relying on the services provided by the cloud accessible via the internet. It introduces distributed storage and parallel computing of Big Data in order to provide robust solutions to the requirements of intensive processing, predictive analysis, and massive storage of learning data. To do this, a methodology is presented and applied which describes the integration process. In addition, this chapter also addresses the deployment of a distributed e-learning architecture combining several recent tools of the Big Data and based on a strategy of data decentralization and the parallelization of the treatments on a cluster of nodes. Finally, this article aims to develop a Big Data solution for online learning platforms based on LMS Moodle. A course recommendation system has been designed and implemented relying on machine learning techniques, to help the learner select the most relevant learning resources according to their interests through the analysis of learning traces. The realization of this system is done using the learning data collected from the ESTenLigne platform and Spark Framework deployed on Hadoop infrastructure

    A Tale of Two Data-Intensive Paradigms: Applications, Abstractions, and Architectures

    Full text link
    Scientific problems that depend on processing large amounts of data require overcoming challenges in multiple areas: managing large-scale data distribution, co-placement and scheduling of data with compute resources, and storing and transferring large volumes of data. We analyze the ecosystems of the two prominent paradigms for data-intensive applications, hereafter referred to as the high-performance computing and the Apache-Hadoop paradigm. We propose a basis, common terminology and functional factors upon which to analyze the two approaches of both paradigms. We discuss the concept of "Big Data Ogres" and their facets as means of understanding and characterizing the most common application workloads found across the two paradigms. We then discuss the salient features of the two paradigms, and compare and contrast the two approaches. Specifically, we examine common implementation/approaches of these paradigms, shed light upon the reasons for their current "architecture" and discuss some typical workloads that utilize them. In spite of the significant software distinctions, we believe there is architectural similarity. We discuss the potential integration of different implementations, across the different levels and components. Our comparison progresses from a fully qualitative examination of the two paradigms, to a semi-quantitative methodology. We use a simple and broadly used Ogre (K-means clustering), characterize its performance on a range of representative platforms, covering several implementations from both paradigms. Our experiments provide an insight into the relative strengths of the two paradigms. We propose that the set of Ogres will serve as a benchmark to evaluate the two paradigms along different dimensions.Comment: 8 pages, 2 figure

    Розробка та дослідження моделей та програмних рішень для рекомендаційної системи вибору товарів масового вжитку

    Get PDF
    There have been proposed investigation of the problem of creating recommendations with technical description for building the Recommender System of consumer goods with help of modern algorithms, approaches, principles and contains the investigation of the most popular methods. It was defined, that the deployment of Recommender Systems is one of the rapidly developing areas for improving applied information technolog ies, tools for automatic generating offers service based on the investigation of the personal needs and profile of customers. It was investigated, that such systems have started to play a very important role in the fast growing Internet, as they help users to navigate in a large amount of information, because users are not able to analyze a large amount of information, because it is very difficult and takes a lot of time and effort, but due to such systems, namely Recommender Systems that are able to filter a large amount of information, and provide for users the information and recommendations their likes the problem can be solved and instead of providing the static information, when users search and, perhaps, buy products, Recommender Systems increase the degree of interactivity to expand the opportunities provided to the user. It was defined, that Recommendation systems form recommendations independently for each specific user based on past purchases and searches, and also on the basis of the behavior of other users with help of recommendation services, which collect different information about a person using several methods and at the same time all systems are shared. An overview of content-based, collaborative filtering and hybrid methods was performed. An overview of Alternating Least Squares and Singular Value Decomposition recommendation algorithms was performed. The design of the Recommender System of consumer goods software component was described. The main features of software implementation and programming tools for the system which is being developed were explained. The conclusions about the problems of Recommender Systems and the review of existing algorithms were made.Запропоновано дослідження проблеми створення рекомендацій, з технічним описом для побудови рекомендаційної системи для вибору товарів масового вжитку за допомогою сучасних алгоритмів, підходів, принципів і містить дослідження найбільш популярних методів. Було визначено, що впровадження рекомендаційних систем є однією з областей, які швидко розвиваються для вдосконалення прикладних інформаційних технологій, інструментів для автоматичного генерування пропозицій, заснованих на дослідженні особистих потреб і профілю клієнтів. Було досліджено, що такі системи почали грати дуже важливу роль в швидко зростаючому Інтернеті, оскільки вони допомагають користувачам орієнтуватися у великій кількості інформації, користувачі не можуть аналізувати великий обсяг інформації, адже це дуже складно і також вимагає багато часу і зусиль, але завдяки рекомендаційним системам, які можуть фільтрувати великий обсяг інформації і надавати користувачам інформацію і рекомендації, які їм подобаються, проблема може бути вирішена і замість надання статичної інформації, коли користувачі шукають, і можливо, купують продукти, такі системи збільшують ступінь інтерактивності для розширення можливостей, що надаються користувачеві. Було визначено, що рекомендаційні системи формують рекомендації самостійно для кожного конкретного користувача на основі минулих покупок і пошуків, а також на основі поведінки інших користувачів за допомогою служб рекомендацій, які збирають різну інформацію про людину, що використовує кілька методів, і в той же час всі системи є загальними. Було проведено огляд методів фільтрації на основі контенту, спільної фільтрації і гібридних методів. Було виконано огляд алгоритмів альтернативних найменших квадратів і сингулярного розкладання. Описана конструкція рекомендаційної системи програмного забезпечення для вибору товарів масового вжитку. Зроблено пояснення деяких можливостей програмної реалізації і інструментів програмування для розроблюваної системи. Зроблено висновки про проблеми рекомендаційних систем і огляд існуючих алгоритмів

    Cloud-based Recommendation Systems: Applications and Solutions

    Get PDF
    Recommender systems have become extremely common in recent years, and are applied in a variety of applications. They help businesses increase their sales and customer satisfaction. More and more computing applications including recommender systems, are being deployed as cloud computing services. This papers presents some of the most common recommendation applications and solutions which follow SaaS, PaaS or other cloud computing service models. They are provided both from academia and business domain and use recent data mining, machine learning and artificial intelligence techniques. The tendency of these kind of applications is towards SaaS service model which seems the most appropriate especially for businesses
    corecore