Ngram-LSTM Open Rate Prediction Model (NLORP) and Error_accuracy@C metric: Simple effective, and easy to implement approach to predict open rates for marketing email

Our generation has seen an exponential increase in digital tools adoption. One of the unique areas where digital tools have made an exponential foray is in the sphere of digital marketing, where goods and services have been extensively promoted through the use of digital advertisements. Following this growth, multiple companies have leveraged multiple apps and channels to display their brand identities to a significantly larger user base. This has resulted in products, worth billions of dollars to be sold online. Emails and push notifications have become critical channels to publish advertisement content, to proactively engage with their contacts. Several marketing tools provide a user interface for marketers to design Email and Push messages for digital marketing campaigns. Marketers are also given a predicted open rate for the entered subject line. For enabling marketers generate targeted subject lines, multiple machine learning techniques have been used in the recent past. In particular, deep learning techniques that have established good effectiveness and efficiency. However, these techniques require a sizable amount of labelled training data in order to get good results. The creation of such datasets, particularly those with subject lines that have a specific theme, is a challenging and time-consuming task. In this paper, we propose a novel Ngram and LSTM-based modeling approach (NLORPM) to predict open rates of entered subject lines that is easier to implement, has low prediction latency, and performs extremely well for sparse data. To assess the performance of this model, we also devise a new metric called 'Error_accuracy@C' which is simple to grasp and fully comprehensible to marketers

The contribution of data mining to information science

The information explosion is a serious challenge for current information institutions. On the other hand, data mining, which is the search for valuable information in large volumes of data, is one of the solutions to face this challenge. In the past several years, data mining has made a significant contribution to the field of information science. This paper examines the impact of data mining by reviewing existing applications, including personalized environments, electronic commerce, and search engines. For these three types of application, how data mining can enhance their functions is discussed. The reader of this paper is expected to get an overview of the state of the art research associated with these applications. Furthermore, we identify the limitations of current work and raise several directions for future research

College Student Retention Behavior: Testing Persuasion with Social Identity Framed Messages on FAFSA Submission

In this study, the experimental trial focused on whether the inclusion of the heuristic cues in email communications to students might affect a larger proportion of the subjects to submit the Free Application for Federal Student Aid (FAFSA) than were observed from a control group. The experiment also evaluated if the inclusion of a heuristic cue within the message affected a subjects likelihood to open the email or click the link provided in the email. The study also considered if gender differences that appear in college student retention and completion outcomes might be present in interactions with email communications. The project failed to discover significant differences from the heuristic cues for FAFSA submissions across three message trials, but significant differences were present in email open and click behaviors, including significant gender differences. The resulting pattern shows message senders and receivers did not follow the same pathway to desired outcomes, even with a clearly defined path. This project affirms that student investment with university email campaigns is not universal and many different heuristic components contribute to a subjects response to a message. Institutions must consider how they communicate with students, including the exploration of multi-modal message distribution, if they want to be sure their messages are heard by the very people they are sending them to, particularly if they want that audience to do as they are told.Keywords: persuasion, email messages, gender differences in higher education, social identity theory, heuristic cues, enrollment management, FAFSA submission, college student retention, college graduatio

Maximizing User Engagement In Short Marketing Campaigns Within An Online Living Lab: A Reinforcement Learning Perspective

ABSTRACT MAXIMIZING USER ENGAGEMENT IN SHORT MARKETING CAMPAIGNS WITHIN AN ONLINE LIVING LAB: A REINFORCEMENT LEARNING PERSPECTIVE by ANIEKAN MICHAEL INI-ABASI August 2021 Advisor: Dr. Ratna Babu Chinnam Major: Industrial & Systems Engineering Degree: Doctor of Philosophy User engagement has emerged as the engine driving online business growth. Many firms have pay incentives tied to engagement and growth metrics. These corporations are turning to recommender systems as the tool of choice in the business of maximizing engagement. LinkedIn reported a 40% higher email response with the introduction of a new recommender system. At Amazon 35% of sales originate from recommendations, while Netflix reports that ‘75% of what people watch is from some sort of recommendation,’ with an estimated business value of $1 billion per year. While the leading companies have been quite successful at harnessing the power of recommenders to boost user engagement across the digital ecosystem, small and medium businesses (SMB) are struggling with declining engagement across many channels as competition for user attention intensifies. The SMBs often lack the technical expertise and big data infrastructure necessary to operationalize recommender systems. The purpose of this study is to explore the methods of building a learning agent that can be used to personalize a persuasive request to maximize user engagement in a data-efficient setting. We frame the task as a sequential decision-making problem, modelled as MDP, and solved using a generalized reinforcement learning (RL) algorithm. We leverage an approach that eliminates or at least greatly reduces the need for massive amounts of training data, thus moving away from a purely data-driven approach. By incorporating domain knowledge from the literature on persuasion into the message composition, we are able to train the RL agent in a sample efficient and operant manner. In our methodology, the RL agent nominates a candidate from a catalog of persuasion principles to drive higher user response and engagement. To enable the effective use of RL in our specific setting, we first build a reduced state space representation by compressing the data using an exponential moving average scheme. A regularized DQN agent is deployed to learn an optimal policy, which is then applied in recommending one (or a combination) of six universal principles most likely to trigger responses from users during the next message cycle. In this study, email messaging is used as the vehicle to deliver persuasion principles to the user. At a time of declining click-through rates with marketing emails, business executives continue to show heightened interest in the email channel owing to higher-than-usual return on investment of$42 for every dollar spent when compared to other marketing channels such as social media. Coupled with the state space transformation, our novel regularized Deep Q-learning (DQN) agent was able to train and perform well based on a few observed users’ responses. First, we explored the average positive effect of using persuasion-based messages in a live email marketing campaign, without deploying a learning algorithm to recommend the influence principles. The selection of persuasion tactics was done heuristically, using only domain knowledge. Our results suggest that embedding certain principles of persuasion in campaign emails can significantly increase user engagement for an online business (and have a positive impact on revenues) without putting pressure on marketing or advertising budgets. During the study, the store had a customer retention rate of 76% and sales grew by a half-million dollars from the three field trials combined. The key assumption was that users are predisposed to respond to certain persuasion principles and learning the right principles to incorporate in the message header or body copy would lead to higher response and engagement. With the hypothesis validated, we set forth to build a DQN agent to recommend candidate actions from a catalog of persuasion principles most likely to drive higher engagement in the next messaging cycle. A simulation and a real live campaign are implemented to verify the proposed methodology. The results demonstrate the agent’s superior performance compared to a human expert and a control baseline by a significant margin (~ up to 300%). As the quest for effective methods and tools to maximize user engagement intensifies, our methodology could help to boost user engagement for struggling SMBs without prohibitive increase in costs, by enabling the targeting of messages (with the right persuasion principle) to the right user

Artificial intelligence applications in marketing: the chatbot of the Department of Economics and Management "Marco Fanno”

openL'intelligenza artificiale (AI) offre numerose applicazioni nel marketing, ma allo stesso tempo ci sono diverse limitazioni da considerare nella sua adozione. Dopo la prima parte di analisi generale delle applicazioni e degli aspetti negativi dell'AI e dei chatbot, la tesi si concentra sul caso dell'implementazione di un chatbot da parte del Dipartimento di Economia e Management “Marco Fanno” dell'Università di Padova. La domanda di ricerca è volta a capire se il chatbot implementato dal Dipartimento sia stato efficace nell'alleggerire e supportare il lavoro dell'ufficio amministrativo e nel rispondere alle domande degli studenti. A tal fine, il documento analizza se il numero di email è diminuito dopo l'introduzione del chatbot. Inoltre è stato svolto un questionario per valutare l'esperienza che gli studenti del Dipartimento hanno avuto con il chatbot di ateneo. Il sondaggio ha anche chiesto agli studenti quali servizi vorrebbero che il chatbot aggiungesse a quelli attuali. Inoltre, è stata condotta un'analisi economica su benefici e costi per valutare se il chatbot genererà un risultato economico positivo. Questo studio consente di valutare l'impatto che un chatbot potrebbe avere nel campo dell'istruzione. In particolare, può fornire informazioni alle università sul fatto che un chatbot possa migliorare il coinvolgimento con gli studenti, liberare il personale da compiti ripetitivi e generare benefici economici netti nel lungo periodo. Il questionario stesso è stato condotto attraverso un sondaggio web su Google Forms e un sondaggio attraverso un chatbot. In questo modo ho anche analizzato quale dei due metodi sia il più efficace per condurre un'indagine. Alcune prove rivelano come i sondaggi condotti attraverso un chatbot possano portare a risposte più accurate da parte degli intervistati. Confrontando i risultati ottenuti della due modalità di sondaggio ho potuto verificare queste evidenze con un nuovo campione di partecipanti, gli studenti di Economia. I risultati della tesi non hanno mostrato prove chiare del fatto che il chatbot consentisse di ridurre il numero di e-mail. Ma si suggerisce un'indagine su un periodo più lungo. Successivamente i risultati hanno evidenziato un buon apprezzamento degli studenti per il chatbot e hanno suggerito l'introduzione di notifiche push che ricordano delle scadenze universitarie come le tasse. La stima dell'analisi costi-benefici prevedeva un risultato netto positivo su tre anni con un ROI del 29%. Inoltre, il sondaggio chatbot ha parzialmente confermato la tendenza ad ottenere risposte più accurate rispetto ad un classico sondaggio web.Artificial intelligence (AI) offers numerous applications in marketing, but at the same time, there are several limitations to consider in its adoption. After the first part about a general analysis of the applications and negative aspects of AI and chatbots, the thesis focuses on the case of the implementation of a chatbot by the Department of Economics and Management “Marco Fanno” of the University of Padua. The research question turns towards understanding whether the chatbot implemented by the Department was effective in easing and supporting the work of the administrative office and answering students questions. For this purpose, the paper analyses if the number of emails is decreased after the chatbot introduction. In addition, a questionnaire was carried out to evaluate the experience that the students of the Department have had with the university chatbot. The survey also asked students what services they would like the chatbot to add to their current ones. Moreover, an economic analysis on benefits and costs was conducted to estimate whether the chatbot will generate a positive outcome. This study allows evaluating the impact a chatbot could have in the education field. In particular, it can provide insight to universities on whether a chatbot could enhance the engagement with students, offload staff from repetitive tasks and generate net economic benefits in the long period. The questionnaire itself was conducted through a web survey on Google Forms and a chatbot survey. In this way, it could also be verified which of the two methods is the most effective to conduct a survey. Some evidence finds how chatbot surveys can lead to less satisfactory answers by respondents. Comparing the two survey results, I can verify these past findings with a different sample of participants, the students of Economics. The results did not show clear evidence of whether the chatbot allowed reducing the number of emails. But an investigation over a longer period is suggested. Then, findings highlighted a good appreciation of students for the chatbot and suggested the introduction of push notifications that remember university deadlines such as taxes. The estimation of the benefits-cost analysis forecasted a net positive outcome over three years with an ROI of 29%. Also, the chatbot survey partially confirmed the encouraging finding in reducing satisficing by respondents.

Empowering email marketing

The purpose of this dissertation is to explore an empowering email marketing strategy that marketers can use for effective, modern email marketing. It describes the strategic transformation of email marketing from one-way persuasive communication to customized two-way interaction using Pettigrew´s (1987) context, content, process (CCP) framework. Consumer empowerment is used as the specific context in which email marketing takes place, and the content and process of email marketing are examined in relation to it. Changes in the business environment, accelerated by the Internet, have shifted the power dynamic between consumers and organizations, transforming their relationship from reactive transaction to proactive collaboration. This has created a need to move beyond persuasive marketing to more interactive and tailored communication. Compared to other interactive marketing practices, such as social media or mobile apps, email seems to be stuck in old, inefficient ways of implementation. Consumers view marketing emails as annoying and irrelevant, even though marketers have better opportunities than ever before to use consumer data to tailor and target messages according to consumer expectations. The research consists of three sub-studies: a systematic literature review using inductive qualitative analysis, and two online controlled experiments using different deductive quantitative analysis methods. It evaluates real-world consumer behavior and seeks to answer the main research question: What are the implications of an organization’s adoption of an empowering email marketing strategy? The dissertation proposes that adopting an empowering email marketing strategy requires advanced first-party data management that enables interaction. Email marketing should be based on permission, and the contents of emails should be tailored to the preferences of the individual recipients, but by directly asking about their preferences rather than inferring them from observed data. According to the study’s empirical findings, content matters: relevant content and active engagement improve behavioral email marketing results (open rates, click-to-open rates, and conversion rates). The study also recommends testing email content in the marketer's own operational environment.Voimaannuttava sähköpostimarkkinointi Väitöskirjassa tutkitaan voimaannuttavan sähköpostimarkkinoinnin strategiaa, jota markkinoijat voivat käyttää tehokkaaseen, nykyaikaiseen markkinointiviestintään. Tutkimus kuvaa sähköpostimarkkinoinnin muutosta yksisuuntaisesta massaviestinnästä räätälöidyksi kaksisuuntaiseksi vuorovaikutukseksi käyttäen viitekehyksenä Pettigrew'n (1987) organisaatiomuutoksen kontekstia, sisältöä ja prosessia kuvaavaa mallia. Kontekstina on kuluttajien voimaantuminen, jonka puitteissa tarkastellaan sähköpostimarkkinoinnin sisältöä ja prosessia. Internetin kiihdyttämät muutokset liiketoimintaympäristössä ovat muuttaneet kuluttajien ja organisaatioiden välisiä valtasuhteita ja tehneet reaktiivisesta vaihdannasta aktiivista yhteistyötä. Muutoksen myötä on tullut tarve siirtyä suostuttelevasta massamarkkinoinnista vuorovaikutteisempaan ja räätälöidympään viestintään. Muihin interaktiivisen markkinoinnin muotoihin, kuten sosiaaliseen mediaan tai mobiilisovelluksiin verrattuna, sähköposti näyttää kuitenkin juuttuneen vanhoihin, tehottomiin toteutustapoihin. Kuluttajat pitävät markkinointisähköposteja ärsyttävinä ja turhina, vaikka markkinoijilla olisi aiempaa paremmat mahdollisuudet käyttää kuluttajatietoja viestien räätälöimiseen ja kohdistamiseen. Tutkimus etenee kolmen osatutkimuksen kautta. Systemaattisessa kirjallisuuskatsauksessa käytetään induktiivista kvalitatiivista analyysiä ja kahdessa koeasetelmassa käytetään deduktiivisia kvantitatiivisia analyysimenetelmiä. Työ arvioi kuluttajien käyttäytymistä todellisessa päätöksentekotilanteessa ja etsii vastausta kysymykseen: Millaisia vaikutuksia voimaannuttavan sähköpostimarkkinointistrategian omaksumisesta on organisaatioille? Väitöskirja esittää, että voimaannuttavan sähköpostimarkkinointistrategian omaksuminen edellyttää kehittynyttä, vuorovaikutuksen mahdollistavaa ensimmäisen osapuolen tiedonhallintaa. Sähköpostimarkkinoinnin tulee perustua lupaan ja sisältöön, joka on räätälöity yksittäisten vastaanottajien mieltymysten mukaan kysymällä suoraan heidän mieltymyksistään havaitun datan hyödyntämisen sijaan. Empiiristen tulosten mukaan uutiskirjeen sisällöllä on väliä: relevantti sisältö ja vuorovaikutus parantavat käyttäytymiseen perustuvia sähköpostimarkkinoinnin tuloksia (avauksia, klikkauksia ja konversioita). Tutkimus suosittelee sähköpostin sisällön testaamista markkinoijan omassa toimintaympäristössä

Let AI Entertain You: Increasing User Engagement with Generative AI and Rejection Sampling

While generative AI excels in content generation, it does not always increase user engagement. This can be attributed to two main factors. First, generative AI generates content without incorporating explicit or implicit feedback about user interactions. Even if the generated content seems to be more informative or well-written, it does not necessarily lead to an increase in user activities, such as clicks. Second, there is a concern with the quality of the content generative AI produces, which often lacks the distinctiveness and authenticity that human-created content possesses. These two factors can lead to content that fails to meet specific needs and preferences of users, ultimately reducing its potential to be engaging. This paper presents a generic framework of how to improve user engagement with generative AI by leveraging user feedback. Our solutions employ rejection sampling, a technique used in reinforcement learning, to boost engagement metrics. We leveraged the framework in the context of email notification subject lines generation for an online social network, and achieved significant engagement metric lift including +1% Session and +0.4% Weekly Active Users. We believe our work offers a universal framework that enhances user engagement with generative AI, particularly when standard generative AI reaches its limits in terms of enhancing content to be more captivating. To the best of our knowledge, this represents an early milestone in the industry's successful use of generative AI to enhance user engagement