Customer churn prediction in telecom using machine learning and social network analysis in big data platform

Customer churn is a major problem and one of the most important concerns for large companies. Due to the direct effect on the revenues of the companies, especially in the telecom field, companies are seeking to develop means to predict potential customer to churn. Therefore, finding factors that increase customer churn is important to take necessary actions to reduce this churn. The main contribution of our work is to develop a churn prediction model which assists telecom operators to predict customers who are most likely subject to churn. The model developed in this work uses machine learning techniques on big data platform and builds a new way of features' engineering and selection. In order to measure the performance of the model, the Area Under Curve (AUC) standard measure is adopted, and the AUC value obtained is 93.3%. Another main contribution is to use customer social network in the prediction model by extracting Social Network Analysis (SNA) features. The use of SNA enhanced the performance of the model from 84 to 93.3% against AUC standard. The model was prepared and tested through Spark environment by working on a large dataset created by transforming big raw data provided by SyriaTel telecom company. The dataset contained all customers' information over 9 months, and was used to train, test, and evaluate the system at SyriaTel. The model experimented four algorithms: Decision Tree, Random Forest, Gradient Boosted Machine Tree "GBM" and Extreme Gradient Boosting "XGBOOST". However, the best results were obtained by applying XGBOOST algorithm. This algorithm was used for classification in this churn predictive model.Comment: 24 pages, 14 figures. PDF https://rdcu.be/budK

Quadri-dimensional approach for data analytics in mobile networks

The telecommunication market is growing at a very fast pace with the evolution of new technologies to support high speed throughput and the availability of a wide range of services and applications in the mobile networks. This has led to a need for communication service providers (CSPs) to shift their focus from network elements monitoring towards services monitoring and subscribers’ satisfaction by introducing the service quality management (SQM) and the customer experience management (CEM) that require fast responses to reduce the time to find and solve network problems, to ensure efficiency and proactive maintenance, to improve the quality of service (QoS) and the quality of experience (QoE) of the subscribers. While both the SQM and the CEM demand multiple information from different interfaces, managing multiple data sources adds an extra layer of complexity with the collection of data. While several studies and researches have been conducted for data analytics in mobile networks, most of them did not consider analytics based on the four dimensions involved in the mobile networks environment which are the subscriber, the handset, the service and the network element with multiple interface correlation. The main objective of this research was to develop mobile network analytics models applied to the 3G packet-switched domain by analysing data from the radio network with the Iub interface and the core network with the Gn interface to provide a fast root cause analysis (RCA) approach considering the four dimensions involved in the mobile networks. This was achieved by using the latest computer engineering advancements which are Big Data platforms and data mining techniques through machine learning algorithms.Electrical and Mining EngineeringM. Tech. (Electrical Engineering

Data mining and predictive analytics application on cellular networks to monitor and optimize quality of service and customer experience

This research study focuses on the application models of Data Mining and Machine Learning covering cellular network traffic, in the objective to arm Mobile Network Operators with full view of performance branches (Services, Device, Subscribers). The purpose is to optimize and minimize the time to detect service and subscriber patterns behaviour. Different data mining techniques and predictive algorithms have been applied on real cellular network datasets to uncover different data usage patterns using specific Key Performance Indicators (KPIs) and Key Quality Indicators (KQI). The following tools will be used to develop the concept: RStudio for Machine Learning and process visualization, Apache Spark, SparkSQL for data and big data processing and clicData for service Visualization. Two use cases have been studied during this research. In the first study, the process of Data and predictive Analytics are fully applied in the field of Telecommunications to efficiently address users’ experience, in the goal of increasing customer loyalty and decreasing churn or customer attrition. Using real cellular network transactions, prediction analytics are used to predict customers who are likely to churn, which can result in revenue loss. Prediction algorithms and models including Classification Tree, Random Forest, Neural Networks and Gradient boosting have been used with an exploratory Data Analysis, determining relationship between predicting variables. The data is segmented in to two, a training set to train the model and a testing set to test the model. The evaluation of the best performing model is based on the prediction accuracy, sensitivity, specificity and the Confusion Matrix on the test set. The second use case analyses Service Quality Management using modern data mining techniques and the advantages of in-memory big data processing with Apache Spark and SparkSQL to save cost on tool investment; thus, a low-cost Service Quality Management model is proposed and analyzed. With increase in Smart phone adoption, access to mobile internet services, applications such as streaming, interactive chats require a certain service level to ensure customer satisfaction. As a result, an SQM framework is developed with Service Quality Index (SQI) and Key Performance Index (KPI). The research concludes with recommendations and future studies around modern technology applications in Telecommunications including Internet of Things (IoT), Cloud and recommender systems.Cellular networks have evolved and are still evolving, from traditional GSM (Global System for Mobile Communication) Circuit switched which only supported voice services and extremely low data rate, to LTE all Packet networks accommodating high speed data used for various service applications such as video streaming, video conferencing, heavy torrent download; and for say in a near future the roll-out of the Fifth generation (5G) cellular networks, intended to support complex technologies such as IoT (Internet of Things), High Definition video streaming and projected to cater massive amount of data. With high demand on network services and easy access to mobile phones, billions of transactions are performed by subscribers. The transactions appear in the form of SMSs, Handovers, voice calls, web browsing activities, video and audio streaming, heavy downloads and uploads. Nevertheless, the stormy growth in data traffic and the high requirements of new services introduce bigger challenges to Mobile Network Operators (NMOs) in analysing the big data traffic flowing in the network. Therefore, Quality of Service (QoS) and Quality of Experience (QoE) turn in to a challenge. Inefficiency in mining, analysing data and applying predictive intelligence on network traffic can produce high rate of unhappy customers or subscribers, loss on revenue and negative services’ perspective. Researchers and Service Providers are investing in Data mining, Machine Learning and AI (Artificial Intelligence) methods to manage services and experience. This research study focuses on the application models of Data Mining and Machine Learning covering network traffic, in the objective to arm Mobile Network Operators with full view of performance branches (Services, Device, Subscribers). The purpose is to optimize and minimize the time to detect service and subscriber patterns behaviour. Different data mining techniques and predictive algorithms will be applied on cellular network datasets to uncover different data usage patterns using specific Key Performance Indicators (KPIs) and Key Quality Indicators (KQI). The following tools will be used to develop the concept: R-Studio for Machine Learning, Apache Spark, SparkSQL for data processing and clicData for Visualization.Electrical and Mining EngineeringM. Tech (Electrical Engineering

Wireless Communications in the Era of Big Data

The rapidly growing wave of wireless data service is pushing against the boundary of our communication network's processing power. The pervasive and exponentially increasing data traffic present imminent challenges to all the aspects of the wireless system design, such as spectrum efficiency, computing capabilities and fronthaul/backhaul link capacity. In this article, we discuss the challenges and opportunities in the design of scalable wireless systems to embrace such a "bigdata" era. On one hand, we review the state-of-the-art networking architectures and signal processing techniques adaptable for managing the bigdata traffic in wireless networks. On the other hand, instead of viewing mobile bigdata as a unwanted burden, we introduce methods to capitalize from the vast data traffic, for building a bigdata-aware wireless network with better wireless service quality and new mobile applications. We highlight several promising future research directions for wireless communications in the mobile bigdata era.Comment: This article is accepted and to appear in IEEE Communications Magazin

Every Moment Counts: Synchrophasors for Distribution Networks with Variable Resources

Chapter 34 in the textbook, "Renewable Energy Integration: Practical Management of Variability, Uncertainty and Flexibility

BERT, SHAP, Kano 모델을 기반한 소비자 만족 요소 다이나믹스

학위논문(석사) -- 서울대학교대학원 : 경영대학 경영학과, 2022.2. 오정석 교수.최근 10년 간 온라인 쇼핑 산업의 성장으로 온라인 쇼핑몰 플랫폼에 온라인 리뷰 등 무한한 소비자 반응, 만족도 관련 정보가 생성되고 있다. 이에 많은 기업들과 학계에서 이를 활용하여 VoC (Voice of Customer)를 반영한 소비자 만족도 모델링을 시도하고 있다. 본 논문은 BERT, GBM, SHAP 등을 활용하여 카노 모델 (Kano Model)에 기반한 소비자 만족도 특성 (Customer Satisfaction Dimension)을 분류하고 각 특성의 소비자 요구 충족 여부가 소비자 만족도에 미치는 영향도를 측정한다. 본 논문의 방법론에 활용된 각 빅데이터 모델 성능과 선행 연구들에서 사용된 모델 성능을 직접 구현 및 비교하여, 본 논문에서 활용된 모델들의 정확성과 안정성을 보였다. 또한 해석적 머신러닝 기법인 SHAP를 도입하여, 카노 카테고리를 분류하는 통일된 분류 방식을 제안한다. 본 연구는 제시된 방법론을 통해 스마트폰 및 스마트워치 제품군을 대상으로 실증 연구를 진행하며, 산업계에 제품 개발 및 개선, 고객 세분화 전략 등 기업 의사결정 방향성에 유의미한 제언을 제시함으로써 본 방법론의 실용적 가치를 입증하였다.As a large number of online reviews are loaded on e-commerce platforms in recent days, companies are being able to measure customer satisfaction reflecting VoC (Voice of Customer) with big data analytics. This paper proposes the improved framework for identifying characteristics of customer satisfaction dimensions (CSD) based on Kano model using BERT (Bidirectional Encoder Representations from Transformers), GBM (Gradient Boosting Machine), and SHAP (Shapley Additive eXplanation). We proved each model outperformance by comparing other models which previous studies have used. And this paper suggests the unified rule of Kano model classification using SHAP. Furthermore, we conducted empirical studies regarding smartphone and smartwatch products which suggest the direction of product enhancement/development strategy and multi-product level customer segmentation strategy to product manufacturers. This shows proposed methodology’s effectiveness and usefulness on industrial analysis.1. Introduction 1 2. A framework for modelling customer satisfaction from online review 5 3. Research Method 8 3.1 Mining customer’s sentiments toward CSDs from online reviews 8 3.2 Measuring the effects of customer sentiments toward each CSD on customer satisfaction 11 3.3 Identifying the feature of each CSD from the customer’s perspective 11 3.4 Classifying each CSD into Kano categories 14 4. Empirical Study 17 4.1 Study 1 17 4.2 Study 2 24 5. Conclusion 27 6. Reference 29석