Balanced Linear Contextual Bandits

Contextual bandit algorithms are sensitive to the estimation method of the outcome model as well as the exploration method used, particularly in the presence of rich heterogeneity or complex outcome models, which can lead to difficult estimation problems along the path of learning. We develop algorithms for contextual bandits with linear payoffs that integrate balancing methods from the causal inference literature in their estimation to make it less prone to problems of estimation bias. We provide the first regret bound analyses for linear contextual bandits with balancing and show that our algorithms match the state of the art theoretical guarantees. We demonstrate the strong practical advantage of balanced contextual bandits on a large number of supervised learning datasets and on a synthetic example that simulates model misspecification and prejudice in the initial training data.Comment: AAAI 2019 Oral Presentation. arXiv admin note: substantial text overlap with arXiv:1711.0707

From Personalized Medicine to Population Health: A Survey of mHealth Sensing Techniques

Mobile Sensing Apps have been widely used as a practical approach to collect behavioral and health-related information from individuals and provide timely intervention to promote health and well-beings, such as mental health and chronic cares. As the objectives of mobile sensing could be either \emph{(a) personalized medicine for individuals} or \emph{(b) public health for populations}, in this work we review the design of these mobile sensing apps, and propose to categorize the design of these apps/systems in two paradigms -- \emph{(i) Personal Sensing} and \emph{(ii) Crowd Sensing} paradigms. While both sensing paradigms might incorporate with common ubiquitous sensing technologies, such as wearable sensors, mobility monitoring, mobile data offloading, and/or cloud-based data analytics to collect and process sensing data from individuals, we present a novel taxonomy system with two major components that can specify and classify apps/systems from aspects of the life-cycle of mHealth Sensing: \emph{(1) Sensing Task Creation \& Participation}, \emph{(2) Health Surveillance \& Data Collection}, and \emph{(3) Data Analysis \& Knowledge Discovery}. With respect to different goals of the two paradigms, this work systematically reviews this field, and summarizes the design of typical apps/systems in the view of the configurations and interactions between these two components. In addition to summarization, the proposed taxonomy system also helps figure out the potential directions of mobile sensing for health from both personalized medicines and population health perspectives.Comment: Submitted to a journal for revie

Enabling Automated, Conversational Health Coaching with Human-Centered Artificial Intelligence

Health coaching is a promising approach to support self-management of chronic conditions like type 2 diabetes; however, there aren’t enough coaching practitioners to support those in need. Advances in Artificial Intelligence (AI) and Machine Learning (ML) have the potential to enable innovative, automated health coaching interventions, but important gaps remain in applying AI and ML to coaching interventions. This thesis aims to identify computational approaches and interactive technologies that enable automated health coaching systems. First, I utilized computational approaches that leverage individuals’ self-tracking and health data and used an expert system to translate ML inferences into personalized nutrition goal recommendations. The system, GlucoGoalie, was evaluated in multiple studies including a 4-week deployment study which demonstrated the feasibility of the approach. Second, I compared human-powered and automated/chatbot approaches to health coaching in a 3-week study which found that t2.coach — a scripted, theoretically-grounded chatbot designed through an iterative, user-centered process — cultivated a coach-like experience that had many similarities to the experience of messaging with actual health coaches, and outlined directions for automated, conversational coaching interventions. Third, I examined multiple AI approaches to enable micro-coaching dialogs — brief coaching conversations related to specific meals, to support achievement of nutrition goals — including a knowledge-based system for natural language understanding, and a data-driven, reinforcement learning approach for dialog management. Together, the results of these studies contribute methods and insights that take steps towards more intelligent conversational coaching systems, with resonance to research in informatics, human-computer interaction, and health coaching