68 research outputs found
Decision-Aware Actor-Critic with Function Approximation and Theoretical Guarantees
Actor-critic (AC) methods are widely used in reinforcement learning (RL) and
benefit from the flexibility of using any policy gradient method as the actor
and value-based method as the critic. The critic is usually trained by
minimizing the TD error, an objective that is potentially decorrelated with the
true goal of achieving a high reward with the actor. We address this mismatch
by designing a joint objective for training the actor and critic in a
decision-aware fashion. We use the proposed objective to design a generic, AC
algorithm that can easily handle any function approximation. We explicitly
characterize the conditions under which the resulting algorithm guarantees
monotonic policy improvement, regardless of the choice of the policy and critic
parameterization. Instantiating the generic algorithm results in an actor that
involves maximizing a sequence of surrogate functions (similar to TRPO, PPO)
and a critic that involves minimizing a closely connected objective. Using
simple bandit examples, we provably establish the benefit of the proposed
critic objective over the standard squared error. Finally, we empirically
demonstrate the benefit of our decision-aware actor-critic framework on simple
RL problems.Comment: 44 page
Case Studies of Causal Discovery from IT Monitoring Time Series
Information technology (IT) systems are vital for modern businesses, handling
data storage, communication, and process automation. Monitoring these systems
is crucial for their proper functioning and efficiency, as it allows collecting
extensive observational time series data for analysis. The interest in causal
discovery is growing in IT monitoring systems as knowing causal relations
between different components of the IT system helps in reducing downtime,
enhancing system performance and identifying root causes of anomalies and
incidents. It also allows proactive prediction of future issues through
historical data analysis. Despite its potential benefits, applying causal
discovery algorithms on IT monitoring data poses challenges, due to the
complexity of the data. For instance, IT monitoring data often contains
misaligned time series, sleeping time series, timestamp errors and missing
values. This paper presents case studies on applying causal discovery
algorithms to different IT monitoring datasets, highlighting benefits and
ongoing challenges.Comment: Accepted to the UAI 2023 Workshop on The History and Development of
Search Methods for Causal Structur
Identification and Modeling Social Media Influence Pathways: a Characterization of a Disinformation Campaign Using the Flooding-the-zone Strategy via Transfer Entropy
The internet has made it easy for narratives to spread quickly and widely without regard for accuracy or the harm they may cause to society. Unfortunately, this has led to the rise of bad actors who use fake and misleading articles to spread harmful misinformation. These actors flood the information space with low-quality articles in an effort to disrupt opposing narratives, sow confusion, and discourage the pursuit of truth. In societies that prioritize free speech, maintaining control over the information space remains a persistent challenge. Achieving this requires strategic planning to protect the dissemination of information in ways that promote dialogue towards organic consensus building and protect users from undue manipulation from foreign adversarial state actors. The objective of this dissertation is to investigate how bad actors can manipulate the information space in societies that value free speech. To achieve this objective, we will define the different narratives used to flood the information space, identify the controversial elements that contribute to their spread, and analyze the actors involved in promoting these narratives and their levels of influence. To gain a deeper understanding of the dynamics that underlie information space flooding, we will examine the flow of influence from news organizations to online users across multiple social networks, and explore the formation of online communities and echo chambers that align with specific narratives. We will also investigate the role of controversiality in information and influence spread, specifically examining how controversial authors tend to be sources of influence in these networks. By addressing these objectives, we hope to provide an analysis of the ways in which bad actors can manipulate the information space. Furthermore, we aim to provide insights into how we can develop strategies to counteract these efforts and protect the integrity of the information ecosystem. Through our investigation, we hope to contribute to the growing body of research focused on understanding and addressing the challenges posed by bad actors in the information space
Hitting the High-Dimensional Notes: An ODE for SGD learning dynamics on GLMs and multi-index models
We analyze the dynamics of streaming stochastic gradient descent (SGD) in the
high-dimensional limit when applied to generalized linear models and
multi-index models (e.g. logistic regression, phase retrieval) with general
data-covariance. In particular, we demonstrate a deterministic equivalent of
SGD in the form of a system of ordinary differential equations that describes a
wide class of statistics, such as the risk and other measures of
sub-optimality. This equivalence holds with overwhelming probability when the
model parameter count grows proportionally to the number of data. This
framework allows us to obtain learning rate thresholds for stability of SGD as
well as convergence guarantees. In addition to the deterministic equivalent, we
introduce an SDE with a simplified diffusion coefficient (homogenized SGD)
which allows us to analyze the dynamics of general statistics of SGD iterates.
Finally, we illustrate this theory on some standard examples and show numerical
simulations which give an excellent match to the theory.Comment: Preliminary versio
Online Bootstrap Inference with Nonconvex Stochastic Gradient Descent Estimator
In this paper, we investigate the theoretical properties of stochastic
gradient descent (SGD) for statistical inference in the context of nonconvex
optimization problems, which have been relatively unexplored compared to convex
settings. Our study is the first to establish provable inferential procedures
using the SGD estimator for general nonconvex objective functions, which may
contain multiple local minima.
We propose two novel online inferential procedures that combine SGD and the
multiplier bootstrap technique. The first procedure employs a consistent
covariance matrix estimator, and we establish its error convergence rate. The
second procedure approximates the limit distribution using bootstrap SGD
estimators, yielding asymptotically valid bootstrap confidence intervals. We
validate the effectiveness of both approaches through numerical experiments.
Furthermore, our analysis yields an intermediate result: the in-expectation
error convergence rate for the original SGD estimator in nonconvex settings,
which is comparable to existing results for convex problems. We believe this
novel finding holds independent interest and enriches the literature on
optimization and statistical inference
Constructing a meta-learner for unsupervised anomaly detection
Unsupervised anomaly detection (AD) is critical for a wide range of practical applications,
from network security to health and medical tools. Due to the diversity of problems, no single algorithm
has been found to be superior for all AD tasks. Choosing an algorithm, otherwise known as the Algorithm
Selection Problem (ASP), has been extensively examined in supervised classification problems, through the
use of meta-learning and AutoML, however, it has received little attention in unsupervised AD tasks. This
research proposes a new meta-learning approach that identifies an appropriate unsupervised AD algorithm
given a set of meta-features generated from the unlabelled input dataset. The performance of the proposed
meta-learner is superior to the current state of the art solution. In addition, a mixed model statistical analysis
has been conducted to examine the impact of the meta-learner components: the meta-model, meta-features,
and the base set of AD algorithms, on the overall performance of the meta-learner. The analysis was
conducted using more than 10,000 datasets, which is significantly larger than previous studies. Results
indicate that a relatively small number of meta-features can be used to identify an appropriate AD algorithm,
but the choice of a meta-model in the meta-learner has a considerable impac
Artificial Intelligence for Online Review Platforms - Data Understanding, Enhanced Approaches and Explanations in Recommender Systems and Aspect-based Sentiment Analysis
The epoch-making and ever faster technological progress provokes disruptive changes and poses pivotal challenges for individuals and organizations. In particular, artificial intelligence (AI) is a disruptive technology that offers tremendous potential for many fields such as information systems and electronic commerce. Therefore, this dissertation contributes to AI for online review platforms aiming at enabling the future for consumers, businesses and platforms by unveiling the potential of AI. To achieve this goal, the dissertation investigates six major research questions embedded in the triad of data understanding of online consumer reviews, enhanced approaches and explanations in recommender systems and aspect-based sentiment analysis
The Challenges of Big Data - Contributions in the Field of Data Quality and Artificial Intelligence Applications
The term "big data" has been characterized by challenges regarding data volume, velocity, variety and veracity. Solving these challenges requires research effort that fits the needs of big data. Therefore, this cumulative dissertation contains five paper aiming at developing and applying AI approaches within the field of big data as well as managing data quality in big data
A Survey on Cross-domain Recommendation: Taxonomies, Methods, and Future Directions
Traditional recommendation systems are faced with two long-standing
obstacles, namely, data sparsity and cold-start problems, which promote the
emergence and development of Cross-Domain Recommendation (CDR). The core idea
of CDR is to leverage information collected from other domains to alleviate the
two problems in one domain. Over the last decade, many efforts have been
engaged for cross-domain recommendation. Recently, with the development of deep
learning and neural networks, a large number of methods have emerged. However,
there is a limited number of systematic surveys on CDR, especially regarding
the latest proposed methods as well as the recommendation scenarios and
recommendation tasks they address. In this survey paper, we first proposed a
two-level taxonomy of cross-domain recommendation which classifies different
recommendation scenarios and recommendation tasks. We then introduce and
summarize existing cross-domain recommendation approaches under different
recommendation scenarios in a structured manner. We also organize datasets
commonly used. We conclude this survey by providing several potential research
directions about this field
Intelligent Sensors for Human Motion Analysis
The book, "Intelligent Sensors for Human Motion Analysis," contains 17 articles published in the Special Issue of the Sensors journal. These articles deal with many aspects related to the analysis of human movement. New techniques and methods for pose estimation, gait recognition, and fall detection have been proposed and verified. Some of them will trigger further research, and some may become the backbone of commercial systems
- …