A Comparison of Machine Learning Models to Prioritise Emails using Emotion Analysis for Customer Service Excellence

There has been little research on machine learning for email prioritization for customer service excellence. To fill this gap, we propose and assess the efficacy of various machine learning techniques for classifying emails into three degrees of priority: high, low, and neutral, based on the emotions inherent in the email content. It is predicted that after emails are classified into those three categories, recipients will be able to respond to emails more efficiently and provide better customer service. We use the NRC Emotion Lexicon to construct a labeled email dataset of 517,401 messages for our proposal. Following that, we train and test four prominent machine learning models, MNB, SVM, LogR, and RF, and an Ensemble of MNB, LSVC, and RF classifiers, on the labeled dataset. Our main findings suggest that machine learning may be used to classify emails based on their emotional content. However, some models outperform others. During the testing phase, we also discovered that the LogR and LSVC models performed the best, with an accuracy of 72%, while the MNB classifier performed the poorest. Furthermore, classification performance differed depending on whether the dataset was balanced or imbalanced. We conclude that machine learning models that employ emotions for email classification are a promising avenue that should be explored further

Kombinasi K-Means dan Support Vector Machine (SVM) untuk Memprediksi Unsur Sara pada Tweet

Tulisan yang disampaikan melalui twitter dinamakan dengan tweets atau dalam bahasa indonesia lebih dikenal dengan kicau, tulisan yang dishare memiliki batas maksimum, tulisan tidak boleh lebih dari 140 karakter, karakter disini terdiri dari huruf, angka, dan simbol. Penyalahgunaan dalam berpendapat sering terjadi di media sosial, sering kali pengguna media sosial dengan sadar atau tidak sadar telah membuat konten yang mengandung isu Suku (dalam hal ini menyangkut keturunan), agama, ras (kebangsaan) dan antargolongan (SARA). Perlu adanya analisis yang dapat mengidentifikasi secara otomatis apakah kalimat yang ditulis pada media sosial mengandung unsur SARA atau tidak, akan tetapi korpus tentang kalimat yang mengandung unsur SARA belum ada, selain itu label kalimat yang menandakan kalimat SARA atau bukan tidak ada. Penelitian ini bertujuan untuk membuat corpus kalimat yang mengandung unsur SARA yang didapatkan dari twitter, kemudian melabeli kalimat dengan label mengandung unsur SARA dan tidak,  serta melakukan sentiment klasifikasi.  Algoritme yang digunakan untuk proses pelabelan adalah k-means, sedangkan Support Vector Machine (SVM) digunakan untuk proses klasifikasi. Hasil yang diperoleh berdasarkan k-means antara lain 118 tweet positif SARA dan 83 tweet negatif SARA. Dalam proses klasifikasi menggunakan dua metode validasi, yaitu 5-fold cross validation yang dibandingkan dengan 10-fold cross validation, hasil akurasi dari kedua metode validasi tersebut yaitu, masing-masing 64,18% dan 63,68%. Berdasarkan hasil akurasi yang diperoleh untuk meningkatkan hasil akurasi, data hasil proses k-means diolah kembali dengan validasi pakar bahasa, hasil yang diperoleh menjadi 139 tweet positif SARA dan 62 tweet negatif SARA, hasil akurasi meningkat menjadi 70,15% dan 71,14%. Dari hasil yang didapatkan, twitter dapat dijadikan sumber untuk membuat corpus mengenai kalimat SARA, dan metode yang diusulkan berhasil untuk proses pelabelan dan sentimen klasifikasi, akan tetapi masih perlu peningkatan hasil akurasi. AbstractPosts sent via twitter are called tweets or in Indonesian better known as chirping, the posts shared have a maximum limit, the writing cannot be more than 140 characters, the characters here consist of letters, numbers, and symbols. Broadcasting in discussions that often occur on social media, often users of social media consciously or unconsciously have created content that contains issues of ethnicity, religion, race (nationality) and intergroup (SARA). Obtained from the analysis that can automatically contain sentences on social media containing no SARA or not, but the corpus about sentences containing SARA does not yet exist, other than that the sentence label indicates SARA or no sentence. This study aims to make sentence corpus containing SARA elements obtained from twitter, then label sentences with labels containing elements of SARA and not, and conduct group sentiments. The algorithm used for the labeling process is k-means, while Support Vector Machine (SVM) is used for the classification process. The results obtained based on k-means include 118 positive SARA tweets and 83 negative SARA tweets. In the classification process using two validation methods, namely cross-fold validation of 5 times compared with 10-fold cross validation, the accuracy of the two validation methods is 64.18% and 63.68%, respectively. Based on the results obtained to improve the results, the k-means process data were reprocessed with linguists, the results obtained were 139 positive SARA tweets and 62 SARA negative tweets, the results of which increased to 70.15% and 71.14%. From the results obtained, Twitter can be used as a source to create a corpus about SARA sentences, and methods that have succeeded in labeling and classification sentiments, but still need to improve the results of accuracy