35 research outputs found
Recommended from our members
Modeling the Dynamics of Consumer Behavior from Massive Interaction Data
Recent technological innovations (e.g. e-commerce platforms, automated retail stores) have enabled dramatic changes in people's shopping experiences, as well as the accessibility to incredible volumes of consumer-product interaction data. As a result, machine learning (ML) systems can be widely developed to help people navigate relevant information and make decisions. Traditional ML systems have achieved great success on various well-defined problems such as speech recognition and facial recognition. Unlike these tasks where datasets and objectives are clearly benchmarked, modeling consumer behavior can be rather complicated; for example, consumer activities can be affected by real-time shopping contexts, collected interaction data can be noisy and biased, interests from multiple parties (both consumers and producers) can be involved in the predictive objectives.The primary goal of this dissertation is to address the obstacles in modeling consumer activities through computational approaches, but with careful considerations from economic and societal perspectives. Intellectually, such models help us to understand the forces that guide consumer behavior. Methodologically, I build algorithms capable of processing massive interaction datasets by connecting well-developed ML techniques and well-established economic theories. Practically, my work has applications ranging from recommender systems, e-commerce and business intelligence
S3-DST: Structured Open-Domain Dialogue Segmentation and State Tracking in the Era of LLMs
The traditional Dialogue State Tracking (DST) problem aims to track user
preferences and intents in user-agent conversations. While sufficient for
task-oriented dialogue systems supporting narrow domain applications, the
advent of Large Language Model (LLM)-based chat systems has introduced many
real-world intricacies in open-domain dialogues. These intricacies manifest in
the form of increased complexity in contextual interactions, extended dialogue
sessions encompassing a diverse array of topics, and more frequent contextual
shifts. To handle these intricacies arising from evolving LLM-based chat
systems, we propose joint dialogue segmentation and state tracking per segment
in open-domain dialogue systems. Assuming a zero-shot setting appropriate to a
true open-domain dialogue system, we propose S3-DST, a structured prompting
technique that harnesses Pre-Analytical Recollection, a novel grounding
mechanism we designed for improving long context tracking. To demonstrate the
efficacy of our proposed approach in joint segmentation and state tracking, we
evaluate S3-DST on a proprietary anonymized open-domain dialogue dataset, as
well as publicly available DST and segmentation datasets. Across all datasets
and settings, S3-DST consistently outperforms the state-of-the-art,
demonstrating its potency and robustness the next generation of LLM-based chat
systems
PEARL: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrievers
Powerful large language models have facilitated the development of writing
assistants that promise to significantly improve the quality and efficiency of
composition and communication. However, a barrier to effective assistance is
the lack of personalization in LLM outputs to the author's communication style
and specialized knowledge. In this paper, we address this challenge by
proposing PEARL, a retrieval-augmented LLM writing assistant personalized with
a generation-calibrated retriever. Our retriever is trained to select historic
user-authored documents for prompt augmentation, such that they are likely to
best personalize LLM generations for a user request. We propose two key
novelties for training our retriever: 1) A training data selection method that
identifies user requests likely to benefit from personalization and documents
that provide that benefit; and 2) A scale-calibrating KL-divergence objective
that ensures that our retriever closely tracks the benefit of a document for
personalized generation. We demonstrate the effectiveness of PEARL in
generating personalized workplace social media posts and Reddit comments.
Finally, we showcase the potential of a generation-calibrated retriever to
double as a performance predictor and further improve low-quality generations
via LLM chaining.Comment: Pre-print, work in progres
Using Large Language Models to Generate, Validate, and Apply User Intent Taxonomies
Log data can reveal valuable information about how users interact with web
search services, what they want, and how satisfied they are. However, analyzing
user intents in log data is not easy, especially for new forms of web search
such as AI-driven chat. To understand user intents from log data, we need a way
to label them with meaningful categories that capture their diversity and
dynamics. Existing methods rely on manual or ML-based labeling, which are
either expensive or inflexible for large and changing datasets. We propose a
novel solution using large language models (LLMs), which can generate rich and
relevant concepts, descriptions, and examples for user intents. However, using
LLMs to generate a user intent taxonomy and apply it to do log analysis can be
problematic for two main reasons: such a taxonomy is not externally validated,
and there may be an undesirable feedback loop. To overcome these issues, we
propose a new methodology with human experts and assessors to verify the
quality of the LLM-generated taxonomy. We also present an end-to-end pipeline
that uses an LLM with human-in-the-loop to produce, refine, and use labels for
user intent analysis in log data. Our method offers a scalable and adaptable
way to analyze user intents in web-scale log data with minimal human effort. We
demonstrate its effectiveness by uncovering new insights into user intents from
search and chat logs from Bing