Maximizing User Engagement In Short Marketing Campaigns Within An Online Living Lab: A Reinforcement Learning Perspective

ABSTRACT MAXIMIZING USER ENGAGEMENT IN SHORT MARKETING CAMPAIGNS WITHIN AN ONLINE LIVING LAB: A REINFORCEMENT LEARNING PERSPECTIVE by ANIEKAN MICHAEL INI-ABASI August 2021 Advisor: Dr. Ratna Babu Chinnam Major: Industrial & Systems Engineering Degree: Doctor of Philosophy User engagement has emerged as the engine driving online business growth. Many firms have pay incentives tied to engagement and growth metrics. These corporations are turning to recommender systems as the tool of choice in the business of maximizing engagement. LinkedIn reported a 40% higher email response with the introduction of a new recommender system. At Amazon 35% of sales originate from recommendations, while Netflix reports that ‘75% of what people watch is from some sort of recommendation,’ with an estimated business value of $1 billion per year. While the leading companies have been quite successful at harnessing the power of recommenders to boost user engagement across the digital ecosystem, small and medium businesses (SMB) are struggling with declining engagement across many channels as competition for user attention intensifies. The SMBs often lack the technical expertise and big data infrastructure necessary to operationalize recommender systems. The purpose of this study is to explore the methods of building a learning agent that can be used to personalize a persuasive request to maximize user engagement in a data-efficient setting. We frame the task as a sequential decision-making problem, modelled as MDP, and solved using a generalized reinforcement learning (RL) algorithm. We leverage an approach that eliminates or at least greatly reduces the need for massive amounts of training data, thus moving away from a purely data-driven approach. By incorporating domain knowledge from the literature on persuasion into the message composition, we are able to train the RL agent in a sample efficient and operant manner. In our methodology, the RL agent nominates a candidate from a catalog of persuasion principles to drive higher user response and engagement. To enable the effective use of RL in our specific setting, we first build a reduced state space representation by compressing the data using an exponential moving average scheme. A regularized DQN agent is deployed to learn an optimal policy, which is then applied in recommending one (or a combination) of six universal principles most likely to trigger responses from users during the next message cycle. In this study, email messaging is used as the vehicle to deliver persuasion principles to the user. At a time of declining click-through rates with marketing emails, business executives continue to show heightened interest in the email channel owing to higher-than-usual return on investment of$42 for every dollar spent when compared to other marketing channels such as social media. Coupled with the state space transformation, our novel regularized Deep Q-learning (DQN) agent was able to train and perform well based on a few observed users’ responses. First, we explored the average positive effect of using persuasion-based messages in a live email marketing campaign, without deploying a learning algorithm to recommend the influence principles. The selection of persuasion tactics was done heuristically, using only domain knowledge. Our results suggest that embedding certain principles of persuasion in campaign emails can significantly increase user engagement for an online business (and have a positive impact on revenues) without putting pressure on marketing or advertising budgets. During the study, the store had a customer retention rate of 76% and sales grew by a half-million dollars from the three field trials combined. The key assumption was that users are predisposed to respond to certain persuasion principles and learning the right principles to incorporate in the message header or body copy would lead to higher response and engagement. With the hypothesis validated, we set forth to build a DQN agent to recommend candidate actions from a catalog of persuasion principles most likely to drive higher engagement in the next messaging cycle. A simulation and a real live campaign are implemented to verify the proposed methodology. The results demonstrate the agent’s superior performance compared to a human expert and a control baseline by a significant margin (~ up to 300%). As the quest for effective methods and tools to maximize user engagement intensifies, our methodology could help to boost user engagement for struggling SMBs without prohibitive increase in costs, by enabling the targeting of messages (with the right persuasion principle) to the right user

An Application Programming Interface (API) framework for digital government

The digital transformation of society obliges governments to evolve towards increasingly complex digital environments. These environments require strong coordination efforts to ensure a synergistic integration of different systems and actors. Application programming interfaces (APIs) are the connective nodes of digital components and thus instrumental enablers of this integration. Yet today, the integration of digital components is often done on an ‘ad hoc’ basis, disregarding the potential value-added for the whole digital environment. This work proposes a framework for a cohesive API adoption in government environments. The framework is distilled from the analysis of an extensive literature review conducted on government API adoption practices to date. The output of this analysis identifies actions to be taken by governments to improve their API infrastructure and related organisational processes. The framework offers 12 ‘proposals’ arranged around four organisational pillars, namely, policy support, platforms and ecosystems, people, and processes. Actions are then organised into the three levels of organisational management, i.e. strategic, tactical and operational. Motivations, implementation details and a self-assessment checklist are provided for each of the proposals. Given that the maturity of digital government structures is uneven, the framework has been designed to be flexible enough to help governments identify the specific actions they need to focus on. The work outlines the basis of an API maturity assessment tool.JRC.B.6-Digital Econom

Mining complex trees for hidden fruit : a graph–based computational solution to detect latent criminal networks : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Information Technology at Massey University, Albany, New Zealand.

The detection of crime is a complex and difficult endeavour. Public and private organisations – focusing on law enforcement, intelligence, and compliance – commonly apply the rational isolated actor approach premised on observability and materiality. This is manifested largely as conducting entity-level risk management sourcing ‘leads’ from reactive covert human intelligence sources and/or proactive sources by applying simple rules-based models. Focusing on discrete observable and material actors simply ignores that criminal activity exists within a complex system deriving its fundamental structural fabric from the complex interactions between actors - with those most unobservable likely to be both criminally proficient and influential. The graph-based computational solution developed to detect latent criminal networks is a response to the inadequacy of the rational isolated actor approach that ignores the connectedness and complexity of criminality. The core computational solution, written in the R language, consists of novel entity resolution, link discovery, and knowledge discovery technology. Entity resolution enables the fusion of multiple datasets with high accuracy (mean F-measure of 0.986 versus competitors 0.872), generating a graph-based expressive view of the problem. Link discovery is comprised of link prediction and link inference, enabling the high-performance detection (accuracy of ~0.8 versus relevant published models ~0.45) of unobserved relationships such as identity fraud. Knowledge discovery uses the fused graph generated and applies the “GraphExtract” algorithm to create a set of subgraphs representing latent functional criminal groups, and a mesoscopic graph representing how this set of criminal groups are interconnected. Latent knowledge is generated from a range of metrics including the “Super-broker” metric and attitude prediction. The computational solution has been evaluated on a range of datasets that mimic an applied setting, demonstrating a scalable (tested on ~18 million node graphs) and performant (~33 hours runtime on a non-distributed platform) solution that successfully detects relevant latent functional criminal groups in around 90% of cases sampled and enables the contextual understanding of the broader criminal system through the mesoscopic graph and associated metadata. The augmented data assets generated provide a multi-perspective systems view of criminal activity that enable advanced informed decision making across the microscopic mesoscopic macroscopic spectrum

Modelling socio-spatial dynamics from real-time data

This thesis introduces a framework for modelling the social dynamic of an urban landscape from multiple and disparate real-time datasets. It seeks to bridge the gap between artificial simulations of human behaviour and periodic real-world observations. The approach is data-intensive, adopting open-source programmatic and visual analytics. The result is a framework that can rapidly produce contextual insights from samples of real-world human activity – behavioural data traces. The framework can be adopted standalone or integrated with other models to produce a more comprehensive understanding of people-place experiences and how context affects behaviour. The research is interdisciplinary. It applies emerging techniques in cognitive and spatial data sciences to extract and analyse latent information from behavioural data traces located in space and time. Three sources are evaluated: mobile device connectivity to a public Wi-Fi network, readings emitted by an installed mobile app, and volunteered status updates. The outcome is a framework that can sample data about real-world activities at street-level and reveal contextual variations in people-place experiences, from cultural and seasonal conditions that create the ‘social heartbeat’ of a landscape to the arrhythmic impact of abnormal events. By continuously or frequently sampling reality, the framework can become self-calibrating, adapting to developments in land-use potential and cultural influences over time. It also enables ‘opportunistic’ geographic information science: the study of unexpected real-world phenomena as and when they occur. The novel contribution of this thesis is to demonstrate the need to improve understanding of and theories about human-environment interactions by incorporating context-specific learning into urban models of behaviour. The framework presents an alternative to abstract generalisations by revealing the variability of human behaviour in public open spaces, where conditions are uncertain and changeable. It offers the potential to create a closer representation of reality and anticipate or recommend behaviour change in response to conditions as they emerge

Analysis and recommendations for the communication’s strategy and channels of CHIC

The 2019 edition of the China Hardware Innovation Camp (CHIC), in which the author is representing the Geneva School of Business Administration, has seen a growth in the number of teams which raised managerial and promotional questions. The importance to address the communication side of CHIC has grown since the ultimate objective of its founders, Marc Laperrouza and Marius Aeberli, is to scale and deploy it more extensively with an innovative form of open education – Open.CHIC. But, its development is limited by the lack of critical resources (human and time), and the need to address the current CHIC's communication to improve, adapt and then replicate it to Open.CHIC. This report aims to analyze the current state of the communication strategy of CHIC, evaluate its communication channels, and assess how its partners communicate about it. The report also seeks to determine the potential of taking advantage of CHIC's networks to promote the program. The insights and data gathered were confronted with representatives of HES-SO's institutions, students, and CHIC's staff's perspective. It aimed to redefine the goals, opportunities, and barriers to implement a more efficient communication strategy. The various recommendations took into account the issues of resources as well as respecting the main objectives of the staff, i.e., to protect CHIC's core, and they sought to valorize the program. Finally, that chapter questioned the recommendations'application to Open.CHIC. To protect CHIC's core and image from any communication risks and to ease CHIC staff's workload, it is recommended to structure and enforce rules and guidelines in the different institutions. These suggestions aim to give more structured autonomy to the various institutions. The promotion and valorization of CHIC should be based on its values and learning outcomes. A digital communication strategy should be developed, tested, and then adapted to the deployment of Open.CHIC. Social media should be enhanced, and the participation of students in the content creation strengthened, with the integration of a communication and marketing's aspect in the program. A content plan's draft is finally proposed to serve as the basis for the development of a communication strategy. Analysis and recommendations for the communication’s strategy and channels of CHIC. Finally, the suggestions were confronted with Open.CHIC. Many questions arise regarding its development, its implementation in the institutions instead of CHIC's current form, and the necessity to promote it. The different recommendations tried to address CHIC's lack of resources with simple actions to put in place. The various suggestions would require some time-investment to be created and implemented in the short-term. But in the long-term, these resources would ease the staff's workload, protect them, and allow them to focus more on the core of CHIC and the development of Open.CHIC

Accelerating Materials Discovery with Machine Learning

As we enter the data age, ever-increasing amounts of human knowledge are being recorded in machine-readable formats. This has opened up new opportunities to leverage data to accelerate scientific discovery. This thesis focuses on how we can use historical and computational data to aid the discovery and development of new materials. We begin by looking at a traditional materials informatics task -- elucidating the structure-function relationships of high-temperature cuprate superconductors. One of the most significant challenges for materials informatics is the limited availability of relevant data. We propose a simple calibration-based approach to estimate the apical and in-plane copper-oxygen distances from more readily available lattice parameter data to address this challenge for cuprate superconductors. Our investigation uncovers a large, unexplored region of materials space that may yield cuprates with higher critical temperatures. We propose two experimental avenues that may enable this region to be accessed. Computational materials exploration is bottle-necked by our ability to provide input structures to feed our workflows. Whilst \textit{ab-intio} structure identification is possible, it is computationally burdensome and we lack design rules for deciding where to target searches in high-throughput setups. To address this, there is a need to develop tools that suggest promising candidates, enabling automated deployment and increased efficiency. Machine learning models are well suited to this task, however, current approaches typically use hand-engineered inputs. This means that their performance is circumscribed by the intuitions reflected in the chosen inputs. We propose a novel way to formulate the machine learning task as a set regression problem over the elements in a material. We show that our approach leads to higher sample efficiency than other well-established composition-based approaches. Having demonstrated the ability of machine learning to aid in the selection of promising compound compositions, we next explore how useful machine learning might be for identifying fabrication routes. Using a recently released data-mined data set of solid-state synthesis reactions, we design a two-stage model to predict the products of inorganic reactions. We critically explore the performance of this model, showing that whilst the predictions fall short of the accuracy required to be chemically discriminative, the model provides valuable insights into understanding inorganic reactions. Through careful investigation of the model's failure modes, we explore the challenges that remain in the construction of forward inorganic reaction prediction models and suggest some pathways to tackle the identified issues. One of the principal ways that material scientists understand and categorise materials is in terms of their symmetries. Crystal structure prototypes are assigned based on the presence of symmetrically equivalent sites known as Wyckoff positions. We show that a powerful coarse-grained representation of materials structures can be constructed from the Wyckoff positions by discarding information about their coordinates within crystal structures. One of the strengths of this representation is that it maintains the ability of structure-based methods to distinguish polymorphs whilst also allowing combinatorial enumeration akin to composition-based approaches. We construct an end-to-end differentiable model that takes our proposed Wyckoff representation as input. The performance of this approach is examined on a suite of materials discovery experiments showing that it leads to strong levels of enrichment in materials discovery tasks. The research presented in this thesis highlights the promise of applying data-driven workflows and machine learning in materials discovery and development. This thesis concludes by speculating about promising research directions for applying machine learning within materials discovery

Videos in Context for Telecommunication and Spatial Browsing

The research presented in this thesis explores the use of videos embedded in panoramic imagery to transmit spatial and temporal information describing remote environments and their dynamics. Virtual environments (VEs) through which users can explore remote locations are rapidly emerging as a popular medium of presence and remote collaboration. However, capturing visual representation of locations to be used in VEs is usually a tedious process that requires either manual modelling of environments or the employment of specific hardware. Capturing environment dynamics is not straightforward either, and it is usually performed through specific tracking hardware. Similarly, browsing large unstructured video-collections with available tools is difficult, as the abundance of spatial and temporal information makes them hard to comprehend. At the same time, on a spectrum between 3D VEs and 2D images, panoramas lie in between, as they offer the same 2D images accessibility while preserving 3D virtual environments surrounding representation. For this reason, panoramas are an attractive basis for videoconferencing and browsing tools as they can relate several videos temporally and spatially. This research explores methods to acquire, fuse, render and stream data coming from heterogeneous cameras, with the help of panoramic imagery. Three distinct but interrelated questions are addressed. First, the thesis considers how spatially localised video can be used to increase the spatial information transmitted during video mediated communication, and if this improves quality of communication. Second, the research asks whether videos in panoramic context can be used to convey spatial and temporal information of a remote place and the dynamics within, and if this improves users' performance in tasks that require spatio-temporal thinking. Finally, the thesis considers whether there is an impact of display type on reasoning about events within videos in panoramic context. These research questions were investigated over three experiments, covering scenarios common to computer-supported cooperative work and video browsing. To support the investigation, two distinct video+context systems were developed. The first telecommunication experiment compared our videos in context interface with fully-panoramic video and conventional webcam video conferencing in an object placement scenario. The second experiment investigated the impact of videos in panoramic context on quality of spatio-temporal thinking during localization tasks. To support the experiment, a novel interface to video-collection in panoramic context was developed and compared with common video-browsing tools. The final experimental study investigated the impact of display type on reasoning about events. The study explored three adaptations of our video-collection interface to three display types. The overall conclusion is that videos in panoramic context offer a valid solution to spatio-temporal exploration of remote locations. Our approach presents a richer visual representation in terms of space and time than standard tools, showing that providing panoramic contexts to video collections makes spatio-temporal tasks easier. To this end, videos in context are suitable alternative to more difficult, and often expensive solutions. These findings are beneficial to many applications, including teleconferencing, virtual tourism and remote assistance

Potentiel de valorisation d'extraits bioactifs issus de bourgeons d'érable à sucre et d'érable rouge

Tableau d'honneur de la Faculté des études supérieures et postdoctorales, 2018-2019Les résidus provenant de l’activité de l’industrie forestière canadienne sont estimés chaque année à plus de 20 millions de tonnes de matière sèche. Des extraits réalisés à partir des écorces de troncs retrouvés dans ces résidus et présentant d’intéressantes propriétés biologiques ont été présentés comme une opportunité de valorisation des produits forestiers vers des domaines de plus fortes valeurs ajoutées. Pourtant, les branches des arbres éliminées durant des éclaircies de peuplements forestiers ou pendant les campagnes d’élagage portent également d’autres tissus végétaux dont la composition chimique et les propriétés biologiques peuvent s’avérer distinctes de celles des écorces. Notre hypothèse de recherche est que les érables, espèces d’importance économique majeure et très répandues des forêts canadiennes, dont les bourgeons se retrouvent dans les résidus forestiers, peuvent également servir à la production d’ingrédients actifs pour les secteurs de l’agroalimentaire, des produits cosmétiques et même de la santé (médicaments, phyto-médicaments, nutraceutiques). En effet, du fait de leur caractère indifférencié, ces bourgeons pourraient contenir des métabolites différents de ceux des autres tissus. Notre projet de recherche a donc eu pour principal objectif, l’exploration des domaines potentiels de valorisation de produits naturels issus de bourgeons de l’érable à sucre et de l’érable rouge. Du fait de la quasi inexistence de données dans la littérature, notre étude a consisté essentiellement à acquérir des connaissances sur la composition chimique de cette matière végétale et à évaluer les effets biologiques in vitro d’extraits et / ou de molécules issues de bourgeons d’érable. Pour cela, différentes méthodes d’extractions et différents solvants peu toxiques et respectueux de l’environnement ont été envisagés. Des explorations chimiques par Chromatographie sur Couche Mince (CCM), des dosages colorimétriques et des déterminations d’activité antioxydante ont ensuite été utilisés pour caractériser et évaluer l’extrait le plus prometteur. La détermination de la nature chimique des constituants majeurs de ce dernier ainsi que des essais biologiques ont été conduits afin de mesurer son potentiel de valorisation dans divers domaines. Les rendements en extraits secs, la nature et quantité en certains types de composés ainsi que les résultats de tests chimiques d’activité antioxydante ont montré que l’extrait à l’eau chaude de bourgeons d’érable rouge présentait un réel potentiel de valorisation comme antioxydant naturel. L’identification des composés phénoliques contenus dans cet extrait et leur quantification ont permis de révéler une forte présence de gallo-tannins, mais également d’hétérosides de quercétine et de cyanidine qui ont été décrits pour la première fois dans cette espèce. L’exploration des effets de cet extrait sur les neutrophiles humains comme première approche n’a indiqué aucune toxicité ni modification significative de leur viabilité jusqu'à 100 μg/mL. Cependant pour des plus fortes concentrations, l’extrait a montré une capacité à accélérer la mort programmée de ces cellules majeures de l’inflammation et cette activité serait due à certains gallotanins. Cette propriété biologique mis en évidence pour la première fois ouvre le champ à de nombreuses voies de valorisation notamment dans la résolution du processus inflammatoire où la survie des neutrophiles est incontestablement liée au développement de pathologies chroniquesThe residues from the activities of Canadian forest industry are estimated to be more than 20 million tons of dry matter per year. The extractives of trunk barks from these residues have been studied as a way to valorize these non- wood forest products in areas with higher added values based on their interesting biological properties for human health. However, branches of the trees removed during thinning or pruning also carry other plant tissues with chemical composition and biological properties distinct from barks. Our research hypothesis was that maples, the widespread trees with major economic value from Canadian forests, represent a source of important quantities of forest residues containing buds which could be used to produce active ingredients for food industry and healthcare (drugs, phytomedications, and nutraceuticals). Indeed, buds contain important amount of meristems, undifferentiated embryonic tissues that may be rich in some bioactive compounds that are often found only in small quantities in other plant parts. Thus, the main objective of our research project was to explore the potential areas of valorization of natural products derived from sugar and red maple buds No data dealing with the chemical composition of this plant material nor biological effects of extracts and / or molecules derived from maple buds were found in scientific literature. The extracts of maple buds (therefore obtained with not toxic and environment- respectful solvents), were analysed by qualitative approach by Thin Layer Chromatography (TLC) assays for screening of phytochemicals. The quantitative assays and antioxidant activity assessment were used to characterize and evaluate the best promising extract. The major phytochemicals of the latter were identified, along with the biological tests undertaken in order to measure its potential of valorization in several fields The yields of dry matter, the nature and quantity of some potential bioactive compounds and the results on antioxidant activities evaluation showed that the hot water extract of red maple buds has a real potential for development as natural antioxidant. The identification of the major phenolic compounds contained in this extract and their quantification revealed an important concentration of gallo-tannins, along with quercetin and cyanidin glycosides, determined in this study for the first time in this species. Exploration of the effects of water extract from red maple buds on human neutrophils as a first approach indicated no toxicity nor significant modification of their viability up to 100 μg / mL. However, for higher concentrations, the extract showed an ability to accelerate the programmed death of these major cells of inflammation. Further studies revealed that this activity was due to particular gallotannins. This biological property highlighted for the first time opens the field to several ways of valorization especially for the resolution of inflammatory processes in which neutrophil survival is undoubtedly linked to the development of chronic pathologies