An Evaluation of Transfer Learning for Classifying Sales Engagement Emails at Large Scale

This paper conducts an empirical investigation to evaluate transfer learning for classifying sales engagement emails arising from digital sales engagement platforms. Given the complexity of content and context of sales engagement, lack of standardized large corpora and benchmarks, limited labeled examples and heterogenous context of intent, this real-world use case poses both a challenge and an opportunity for adopting a transfer learning approach. We propose an evaluation framework to assess a high performance transfer learning (HPTL) approach in three key areas in addition to commonly used accuracy metrics: 1) effective embeddings and pretrained language model usage, 2) minimum labeled samples requirement and 3) transfer learning implementation strategies. We use in-house sales engagement email samples as the experiment dataset, which includes over 3000 emails labeled as positive, objection, unsubscribe, or not-sure. We discuss our findings on evaluating BERT, ELMo, Flair and GloVe embeddings with both feature-based and fine-tuning approaches and their scalability on a GPU cluster with increasingly larger labeled samples. Our results show that fine-tuning of the BERT model outperforms with as few as 300 labeled samples, but underperforms with fewer than 300 labeled samples, relative to all the feature-based approaches using different embeddings

Situation Awareness and Information Fusion in Sales and Customer Engagement: A Paradigm Shift

With today's savvy and empowered customers, sales requires more judgment and becomes more cognitively intense than ever before. We argue that Situation Awareness (SA) is at the center of effective sales and customer engagement in this new era, and Information Fusion (IF) is the key for developing the next generation of decision support systems for digital and AI transformation, leveraging the ubiquitous virtual presence of sales and customer engagement which provides substantially richer capacity to access information. We propose a vision and path for the paradigm shift from Customer Relationship Management (CRM) to the new paradigm of IF. We argue this new paradigm solves major problems of the current CRM paradigm: (1) it reduces the burden of manual data entry and enables more reliable, comprehensive and up-to-date data and knowledge, (2) it enhances individual and team SA and alleviates information silos with increased knowledge transferability, and (3) it enables a more powerful ecosystem of applications by providing common shared layer of computable knowledge assets.Comment: 2020 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA